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ABSTRACT 


Communication  systems,  in  which  a  noise-like 
signal  is  used  as  an  information  carrier  and  cross¬ 
correlation  at  the  receiver  is  used  for  detection,  are 
investigated-  The  probability  of  error  in  the  recep¬ 
tion  of  signals  by  such  systems  (called  NOMAC  sys¬ 
tems)  is  given  as  a  function  of  input  signal-to-noise 
ratio,  input-to-output  bandwidth  ratio,  and  the  num¬ 
ber  of  possible  signals.  The  effect  of  having  a  noisy 
version  of  the  signal  with  which  the  input  signal  is 
cross-correlated  is  included,  and  the  effect  of  using 
an  arbitrary  threshold  value  of  the  output  as  a  crite¬ 
rion  of  detection  is  shown  to  result  in  a  loss  of  avail¬ 
able  channel  capacity,  and  correspondingly  higher 
probability  of  error.  Proposed  practical  systems 
employing  NOMAC  principles  are  described  in  some 
detail,  along  with  the  experimental  system  that  has 
been  constructed  and  tested.  The  experimental  re¬ 
sults  are  shown  to  agree  with  the  theory. 


♦This  report  is  identical  with  a  thesis  of  the  same  title  submitted  in 
partial  fulfillment  of  the  requirements  for  the  Degree  of  Doctor  of 
Science  in  the  Department  of  Electrical  Engineering  at  the  Massachu¬ 
setts  Institute  of  Technology,  26  May  1952. 
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CHAPTER  I 

THE  BASIC  COMMUNICATION  SYSTEM 
A,  Elements  of  Communication  Theory 

The  theory  of  information  has  occupied  a  position  of  growing  importance  in  commu¬ 
nication  engineering  recently.  The  main  purpose  of  the  theory  is  to  provide  means  for  a  quan¬ 
titative  analysis  of  communication  systems.  How  this  purpose  is  achieved  is  illustrated  by  the 
following  discussion  of  a  basic  communication  system. 

The  communication  process  starts  with  the  selection  at  the  transmitter  of  one  mem¬ 
ber  of  a  set  of  possible  messages,  which  in  the  idealized  model  take  the  form  of  symbols.  (In 
electrical  communication  systems,  a  "symbol"  is  usually  a  voltage  or  current  waveform. )  The 
selection  is  made  at  the  direction  of  the  originator,  and  the  selected  symbol  is  transformed  in¬ 
to  a  form  appropriate  for  transmission  in  the  channel.  The  receiver  function  is  to  indicate 
which  of  the  set  of  possible  symbols  was  selected;  if  this  is  done  correctly,  the  communication 
link  has  performed  its  task  perfectly  (see  F 1.1). 


Fig.  1.1.  Block  diagram  of  general  communication  system. 

The  interpretation  of  me  operation  of  a  communication  system  as  a  process  of 
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selection  of  one  from  a  number  of  possibilities  was  made  by  Nyquist  in  19 24.  This  inter¬ 
pretation  was  later  used  by  Hartley  in  a  1928  paper ^  in  which  the  logarithm  of  the  number  of  * 
possible  symbols  is  suggested  as  a  quantitative  measure  of  the  information  conveyed  by  the 
selection. 

Tha  introduction  of  statistical  concepts  in  information  theory  led  to  a  more  gen¬ 
eral  measure  of  information  in  terms  of  the  logarithm  of  the  reciprocal  of  the  probability  that 


♦Refer  to  numbered  references  at  end  of  report. 
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a  symbol  of  the  set  should  be  selected.  (Obviously,  where  the  set  consists  of  equally  probable 
symbols,  this  measure  becomes  identical  with  that  suggested  by  Hartley.)  An  account  of  this 

phase  of  information  theory  is  contained  in  the  literature,  and  the  reader  is  particularly  directed 

25  ZZ  5 

to  the  writings  of  Wiener.  Shannon.  and  Fano . 

if  noise  is  introduced  in  the  channel,  it  has  the  effect  of  distorting  the  received 
signal  in  such  a  way  that  the  distortion  might  have  resulted  from  more  than  one  of  the  possible 
symbols  at  the  transmitter,  so  far  as  the  receiver  is  concerned.  The  receiver,  therefore,  can¬ 
not  be  certain  about  what  was  transmitted  because  of  the  uncertainty  connected  with  the  disturb¬ 
ing  noise,  and  thus  is  inherently  subject  to  errors  in  the  decoding  process. 

From  the  point  of  view  of  information  theory,  the  information  about  the  transmitted 
symbol  at  the  input  of  the  receiver  can  be  expressed  in  terms  of  the  change  in  the  probabilities 
of  the  possible  symbols  upon  receipt  of  a  signal.  Thus,  where  the  set  X  represents  the  pos¬ 
sible  symbols  at  the  transmitter  and  Z  represents  the  signal  at  the  input  of  the  receiver  when 
one  of  the  X's  is  selected  and  transmitted,  the  information  gain  associated  with  each  of  the 
symbols  is  given  by 

1(X/Z)  r  log^^yL  .  (1-1) 

In  Eq.(l~l)t  P(X)  is  the  probability  of  the  symbol  while  P(X/Z)  is  the  conditional  probability 
of  the  symbol  following  the  reception  of  Z.  The  information  received  about  the  transmitted 
symbol  is  the  average  of  that  shown  in  Eq.(l-l),  averaged  over  all  X' that  could  have  resulted 
in  the  particular  Z  received,  namely, 

I(Z)  -  Sp(X/Z)log  -  (1-2) 

Note  that,  in  the  absence  of  interfering  noise,  one  of  the  P(X/Z)  would  be  unity,  all  others, 
zero.  Then  Eq.(l-2)  reduces  to  the  measure  given  in  the  earlier  paragraph,  i.e,,  logl/P(X), 
which  measures  the  information  associated  with  the  selection  made  at  the  transmitter,,  and  thus 
is  the  measure  of  the  transmitted  information. 

The  point  of  view  of  the  discussion  of  the  preceding  paragraph  is  that  of  Woodward 
?  h 

and  Davies.  They  concluded  that  the  best  a  decoder  or  receiver  can  possibly  do  is  to  compute 
the  conditional  probabilities  of  the  transmitted  symbols  when  a  signal  appears  at  the  receiver 
input.  To  demonstrate  how  this  might  he  done.  Woodward  and  lavies  considered  the  case  where 
the  disturbance  in  the  channel  is  an  independent  additive  Gaussian  white  noise.  They  show  that 
the  conditional  probabilities  are  given  by  a  decreasing  function  of  the  mean  square  difference 
between  the  received  signal  and  the  waveform  representing  the  symbol  for  which  the  probabil¬ 
ity  is  being  computed.  Thus,  for  the  symbol 


fT[Z(t)-X.(t)]2  dt" 

- - N  - -  (]-3' 

o 

Here  B  is  a  normalizing  constant  and  is  the  noise  power  per  cycle  of  bandwidth.  If  the 
X's  are  equally  likely,  pCX^)  is  a  constant  for  all  k,  and  p(X.^/Z)i  is  a  function  only  of  the  in- 
tegral  of  the  squared  difference  [Z(t)  —  (t )] ^ . 


p(X./Z)  =  B  p(X.J  exp 

K  K. 
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If  a  receiver  performs  the  computation  of  the  conditional  probabilities  and  makes 
these  quantities  available  to  the  user  of  the  information,  it  has  relayed  all  the  received  information 
to  the  output  circuit.  An  examination  of  Eq.p -2)  reveals  that  on  the  average  1(2)  is  less  than 
the  transmitted  information  when  noise  is  present.  The  information  lost  is  termed  equivocation. 

It  is  also  apparent  that  a  receiver  that  computes  the  conditional  probabilities  in  the 
manner  described  by  Eq.(l-3)  performs  a  comparison  of  the  received  signal  with  each  of  the 
possible  transmitted  symbols.  It  is  correctly  implied  that  copies  of  the  possible  symbols  must 
be  made  available  at  the  receiver  for  the  comparison. 

One  of  the  problems  in  communication  theory  has  been  how  to  select  the  optimum 
set  of  signals  into  which  the  transmitter  encodes  the  information  so  as  to  lead  to  a  minimum 
amount  of  equivocation  under  the  condition  of  fixed  rate  of  transmitted  information.  This  prob¬ 
lem  has  not  been  solved  in  general.  It  has  been  shown,  however,  that  sets  do  exist  which  can 
lead  to  ratios  of  equivocation  to  transmitted  information  that  are  arbitrarily  small,  if  sufficient 
delay  is  allowed  in  the  communication  process.  This  is  true  provided  the  rate  of  transmission 
of  information  does  not  exceed  the  maximum  rate  at  which  information  may  be  received  through 
any  particular  communication  channel. 

This  fact  was  stated  as  an  existence  theorem  and  proved  by  Shannon.^  *  In  a  par¬ 
ticular  form  that  is  applicable  to  continuously  varying  time  functions  disturbed  by  white  Gaussian 
noise.  Shannon  derived  the  maximum  rate,  called  channel  capacity,  which  is  given  by 

C  =  W  log(  1  +  — )  bits/ sec  ( 1-4) 

In  this  expression,  W  is  the  (ideal  rectangular)  bandwidth  occupied  by  the  time-varying  signal, 

S  is  the  component  of  received  signal  power  due  to  the  transmitted  signal,  and  N  is  the  noise 
component  of  the  received  signal  power.  An  important  condition  leading  to  the  derivation  of 
Eq.(  1 -4)  is  that  the  average  transmitted  power  is  limited. 

In  his  derivation  Shannon  noted  that  each  of  the  waveforms  of  the  set  of  signals 
that  would  lead  to  a  full  utilization  of  the  system  capacity  with  arbitrarily  low  equivocation 
would  be  in  all  respects  similar  to  white  Gaussian  noise.  Although  unable  to  determine  a  par¬ 
ticular  optimum  set  of  noise-like  waveforms,  he  was  able  to  show  that  the  average  performance 
of  all  possible  noise  waveforms  (having  the  same  bandwidth  and  average  power)  was  ideal,  in 
that  the  fraction  of  transmitted  information  lost  could  be  held  arbitrarily  low  at  information 
rates  less  than  the  channel  capacity,  provided  sufficient  delay  were  allowed.  This  performance 
could  be  obtained  with  a  receiver  that  makes  a  decision  about  what  was  transmitted  based  on  the 
minimum  mean  square  difference  between  the  received  signal  and  each  of  the  possible  symbols. 

A  practical  and  fortunate  corollary  is  that  random  segments  of  Gaussian  white  noise  can  be  used 
for  symbols,  and  that  these  segments  of  noise  can  be  taken  from  currently  generated  waveforms 
from  a  continuous  noise  source. 

Here  a  new  concept  of  the  role  played  by  the  decoder  or  receiver  has  been  intro¬ 
duced.  It  has  been  stated  that  the  best  a  receiver  can  do  when  noise  is  present  is  to  compute 
the  conditional  probabilities  of  the  transmitted  symbols  following  reception  of  a  signal.  However, 
in  a  practical  case,  the  receiver  is  usually  called  upon  to  indicate  which  of  the  set  of  possible 
symbols  was  transmitted.  The  indication  is  performed  after  a  decision  by  the  receiver  of  which 
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symbol  should  be  indicated.  Because  o:  the  interfering  noise,  the  receiver  cannot  decide  with 
certainty,  but  must  choose  a  symbol  that  has  at  least  a  high  probability  of  having  been  trans- 
mitted.  In  a  typical  case,  the  receiver  might  indicate  the  most  probable  of  the  transmitted  sym¬ 
bols  (as  Shannon's  receiver  does). 

The  indication  made  by  the  receiver  is,  of  course,  subject  to  errors.  The  ratio 
of  the  number  of  erroneous  decisions  to  total  decisions  is  termed  the  probability  of  error,  and 
is  closely  related  to  the  fraction  of  transmitted  information  that  becomes  lost  -  the  per-unit 
equivocation. 

When  the  receiver  makes  a  decision  about  what  is  transmitted,  and  only  this  deci¬ 
sion  is  relayed  to  the  destination,  the  user  of  the  information  knows  only  that  the  receiver  has 
selected  one  of  the  set  of  possibilities  in  accordance  with  some  detection  criterion.  The  infor¬ 
mation  about  the  conditional  probabilities  is  otherwise  discarded,  and  the  over-ail  per-unit 
equivocation  is  thereby  increased. 

While  the  per-unit  equivocation  is  the  proper  criterion  for  the  evaluation  of  the 
efficiency  with  which  a  communication  link  performs  its  assigned  task,  the  probability  of  error 
is  often  used  instead.  The  relation  between  this  relative  frequency  of  erroneous  decisions  and 
the  per-unit  equivocation  when  the  user  of  information  gets  only  the  decisions  of  the  decoding 
device  (receiver)  is  given  in  Appendix  I.  It  is  shown  that  the  per-unit  equivocation  is  a  decreas¬ 
ing  function  of  decreasing  probability  of  error,  which  justifies  the  use  of  the  latter  in  evaluating 
the  performance  of  the  link. 

B.  The  Mathematical  Model 

The  geometrical  model  employed  by  Shannon.1^  and  which  was  subsequently  used 
by  Rice,  was  also  adopted  for  the  theoretical  work  in  this  paper.  It  is  useful  in  relating  the 
concepts  and  results  found  in  this  research  to  those  found  by  these  two  earlier  authors;  it  is 
outlined  briefly  here. 

In  its  simplest  form,  it  is  assumed  that  the  symbols  used  are  segments  of  a  time 
function  which  has  (  1)  a  Gaussian  amplitude  distribution,  (2)  a  flat  frequency  distribution  to  W 
cycles  per  second  with  no  component  frequencies  higher  than  W  cycles  per  second,  and  (3)  an 
amplitude  variance  S  (which  becomes  the  average  power  for  electrical  signals).  Each  of  these 
segments  lasts  just  T  seconds. 

It  is  obvious  that,  because  of  the  abrupt  start  and  stop  of  the  segments  as  described, 
frequency  components  outside  the  bandwidth  W  cannot  be  avoided.  This  suggests  the  character¬ 
ization  of  each  of  the  segments  in  terms  of  the  Fourier  coefficients  in  the  manner  given  by 


X(t) 


X 


VZTW 


TW 

+  £ 

i=  1 


Vtw 


2wi 

cos  -r  t 


TW  X- 
2  - sin 

j=l  Vtw 


(1-5) 


*For  a  complete  discussion  of  this  relation,  see  Ft.  M.  Fano's  printed  lecture  notes  of  yearly 
course,  6.574,  Statistical  Theory  of  Information,  M.I.  T. 
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Here 


X  .  fT 
— -  =  W  X(t)dt 
V2TW  1  Jo 


'Y  f  X  (t)  cos  dt 

Jo  1 


|'TW  T  Jo 


X. 

1 

.Tw 

X- 


X(t)  sin  t  dt 


(1-Sa) 


(lob) 


(1 -5c) 


The  or  X-  represent  the  Fourier  coefficients,  and  X(t)  is  the  segment  of  noise  represented. 

The  constant  1//TW  is  introduced  arbitrarily  to  control  the  relative  magnitudes  of  the  X's. 

It  can  be  shown  that,  for  the  time  function  as  described,  the  X's  are  numbers  drawn  from  a 
normal  distribution  with  variance  equal  to  the  power  S.  By  setting  an  upper  limit  to  the  fre¬ 
quencies  for  which  are  defined,  the  higher-frequency  components  introduced  by  the  starting 
and  stopping  of  a  segment  are  neglected.  As  developed,  the  time  functions  described  will  be 
low-pass  functions.  However,  the  description  of  band-pass  functions  can  be  achieved  analogously 
by  running  the  index  of  summation  from  i  -  TW^  to  i  =  TW^  where  the  frequency  band  between 
Wj  and  is  the  part  of  the  spectrum  occupied  by  the  function. 

Another  description  is  necessary  to  support  the  material  presented  in  Chapter  VIII. 
Although  it  is  somewhat  easier  to  visualize,  it  is  limited  in  application  to  low-pass  functions. 

It  is  also  useful  in  interpreting  the  vector  model  introduced  later  in  this  chapter.  Here,  each 
segment  is  represented  by  amplitude  samples  spaced  each  1/2 W  seconds  along  the  waveform. 
Thus,  there  are  2TW  samples  for  each  symbol  represented  by  a  finite-duration  noise-like  wave¬ 
form.  Because  the  time  function  has  a  flat  rectangular  spectrum,  it  can  easily  be  shown  that 
the  amplitude  samples  are  incoherent;  and,  because  they  are  from  a  Gaussian  process,  they 
are  in  fact  independent.  These  samples  can  thus  be  used  with  a  sequence  of  orthonormal  func¬ 
tions  of  the  form  sin  t/t  to  reconstruct  an^approximation  to  the  original  waveform.  One  takes 


n  sin  2nW  (t  —  t.) 

x<t>  s  xi 


(1-6) 


in  which  b  -  i/ZW.  Tl^e  X(t)  resulting  from  this  summation  not  only  differs  from  the  true  value 
in  the  intersample  time  regions,  but  differs  from  zero  outside  the  duration  T.  It  is,  however, 
a  least-rnean-square  approximation  to  the  true  value,  as  good  as  can  be  done  with  the  2TW 
specifying  numbers.  It  is  in  that  respect  entirely  equivalent  to  the  representation  in  terms  of 
Fourier  coefficients.  Like  the  Fourier  coefficients,  it  is  apparent  that  the  X:s  in  Eq.(l-6)  are 
also  numbers  chosen  from  a  normal  distribution  of  variance  S. 

The  n  =  2TW*  numbers  thus  chosen  to  represent  each  symbol  are  ordered  and  taken 
as  coordinates  of  a  point  in  Euclidean  n-space.  Each  symbol  waveform  corresponds  uniquely  to 


a  point  in  the  space,  such  that  there  are  K  +  1  of  the  points  designated  by'  X  ,  Xj. 


Xk-  Con¬ 


necting  each  point  and  the  origin  are  K  *  1  vectors  denoted  by  X^.  Either  a  point  cr  a  vector 
may  be  used  to  represent  the  message  waveform  to  which  it  corresponds. 
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It  is  evident  that  any  point  in  n -space  specifies  a  waveform  of  the  same  bandwidth 
and  time  duration  as  the  possible  message  waveforms.  The  sum  of  two  waveforms,  by  summing 
coordinates,  becomes  the  vector  sum  of  the  two  waveform  vectors. 

One  of  these  vectors  (arbitrarily  X^)  is  selected  and  transmitted.  In  the  channel, 
an  independent  white  Gaussian  noise,  is  added.  The  received  signal  is  by  linear  superposi¬ 
tion  the  vector  sum  (see  Fig.  1-2)  of  and  Y  and  is  designated  Z.  The  receiver  used  by  Shannon 

and  Rice  has  available  copies  of  the  set  of  vectors  and  chooses,  as  the  one  most  probably 
sent,  the  one  whose  terminal  point  is  closest  to  point  Z.  Here  it  is  evident  that  the  mean  square 
difference  between  the  received  signal  and  each  member  of  the  set  of  possible  signals  is  equal 
to  the  squared  distance  from  the  point  to  each  point  X^,  aside  from  a  constant  factor,  l/n.  (The 
term  "mean "  in  mean  square  difference  is  used  to  connote  the  average  over  the  duration  T  of 

Since  the  interfering  noise  has  been  assumed 
to  be  white  Gaussian,  the  Woodward  and  Davies  treat¬ 
ment  giving  the  conditional  probabilities  in  terms  of  the 
mean  square  difference  is  valid.  Therefore,  it  follows 
that  the  conditional  probability  of  the  symbol  is  a 
monotonically  decreasing  function  of  increasing  distance 
of  the  point  from  Z. 

The  probability  of  error  for  a  communication 
system,  choosing  as  the  transmitted  waveform  the  one 
that  is  the  minimum  distance  from  the  received  signal 
waveform  (in  terms  of  the  geometrical  model),  was  the 
subject  of  Rice's  paper.  When  modified  for  small  sig¬ 
nal -to -noise  ratios,  in  terms  of  the  channel  capacity 
C  and  information  rate  at  the  source  H  -  log^  (K  +  1),  the  probability  of  error  as  obtained  in 
Appendix  III  is 

P  (error)  - - ^ - expf-C'Ttl  -  £)1  .  (1-7) 

2  l/irCT  L  U  J 

This  result  is  valid  for  large  n,  for  large  number  of  possible  messages,  K  +  1,  and  for  a  sig¬ 
nal- to -noise  ratio  sufficiently  small  that  the  approximation  nS/2N  -  C’T  is  valid.  The  prime 
indicates  C  is  in  logarithmic  units.  In  bits,  C  is  equal  to  [log^ejc'- 

It  is  evident  from  Eq.(l-7)  that  the  probability  of  error  and  thus  the  per-unit  equiv¬ 
ocation  may  be  made  arbitrarily  small  with  increasing  delay  T,  provided  only  that  H/C,  the 
ratio  of  transmitted  rate  of  information  to  system  capacity.,  does  not  exceed  unity. 

Rice  did  not  give  Eq.(l-?)  explicitly  in  his  paper,  but  gave  an  expression  for  the 
probability  of  no  error,  valid  for  large  n  and  large  K.  but  for  all  signal -to- noise  ratios.  He 
showed  that  his  expression  approaches  unity  under  the  conditions  above.  However,  his  expres¬ 
sion  was  given  along  with  error  terms,  which  decreased  with  increasing  n  and  K  but  which  ex¬ 
ceeded  the  difference  between  his  expression  and  unity.  Thus,  a  probabilky-of-error  expres¬ 
sion  could  not  be  obtained  directly  from  his  results,  but  an  intermediate  result  had  to  be  taken 
and  worked  into  the  form  of  Eq.(l-7)  as  is  shown  in  Appendix  111.  The  fact  that  the  expression 


the  difference  [Z(t)  -  X(t)]2.) 


Fig.  1.2  Vector  model  of  mes 
sages,  noise,  and  received  signal* 
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given  here  is  limited  to  eases  where  the  signal-to-noise  ratio  at  the  receiver  is  small  is  not  a 
serious  disadvantage,  as  will  be  seen  later. 

It  was  mentioned  earlier  that  a  receiver  may  use  other  criteria  in  deciding,  after 
a  signal  has  been  received,  which  one  of  the  possible  waveforms  was  transmitted.  When  such 
criteria  are  proposed,  they  must  be  evaluated  in  terms  of  the  probability  of  error  that  accom¬ 
panies  the  use  of  that  criteria.  Of  course,  other  prevailing  conditions,  such  as  the  method  of 
coding  and  type  of  interference,  must  be  given  due  consideration.  In  the  class  of  idealized  sys¬ 
tems  in  which  the  possible  message  waveforms  are  random  samples  of  white  Gaussian  noise  and 
the  interfering  noise  is  also  Gaussian,  the  rate  of  errors  can  be  compared  with  the  result  ob¬ 
tained  by  Rice.  Rice's  results  may  be  regarded  as  expressing  the  performance  of  an  ideal  sys¬ 
tem  in  that  his  receiver  will  always  select  the  most  probable  transmitted  symbol  (under  the  con¬ 
ditions  of  additive  Gaussian  noise  and  equal  symbol  probabilities). 
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CHAPTER  II 


CORRELATION  DETECTION  CRITERIA 


Another  criterion  of  detection  that  is  of  primary  interest  in  this  paper  can  be  ob¬ 
tained  as  follows.  Where  the  various  possible  transmitter  symbols  are  given  by  waveforms 
X,{t),  0  <  t  <  T,  and  the  received  signal  is  Z  (t)  =  X  (t)  +  Y  (t)  (where  Y  (t)  is  additive  noise). 

K  O 

the  minimum  mean  square  difference  (msd)  is  given  by  the  minimum  of 
T 

msd  =y  r  [z(t>  -  *  (t)]zdt  ,  (2-i) 

^ O 


4  v»2 


2Z(t)Xk(t)J  dt 


(2-2) 


If  it  is  assumed  that 
-T 
uo 


i  rT  2 
J  xk  (t)  dt 


is  a  constant  for  all  kf  a  minimum  of  Eq.  (2-2)  is  a  maximum  of  the  term 
-T 
'o 


^  j  Z(t)Xk  (t)dt 


This  may  be  recognized  as  the  "correlation  coefficient"  of  X.  and  Z,  and  a  receiver  that  com- 

K  7 

putes  this  value  for  subsequent  use  in  deciding  what  was  sent  is  called  a  correlation  detector. 

Correlation  techniques  were  introduced  to  communication  engineers  largely  by 

16  1 5 

N.  Wiener,  and  the  techniques  have  been  improved  and  implemented  by  Lee  and  others. 

The  cross-correlation  function  of  two  stationary  random  functions  of  time  is  given  by 


^12(T) 


fxCt)  f2  (t  +  t)  dt 


(2-3) 


When  (t)  is  identically  f.(t),  this  becomes  the  autocorrelation  function.  The  relation  between 
*  1  24 

autocorrelation  functions  and  power  spectra  makes  correlation  techniques  an  invaluable  aid  in 
the  study  of  random  functions. 

6 

The  so-called  "short-time  cross-correlation  function"  has  been  studied  and  is 


given  by 

0t(t) 


fjWfU  +  t)  dt 


(2-4) 


which  is  seen  to  be  of  the  form  of  the  defining  expression  for  correlation  coefficients.  A  gener¬ 
alized  form  of  Eq.  (2-4)  in  which  the  product  f ^  (t)  f^  (t  +  t)  is  filtered  rather  than  mathematically 
integrated  is 


<p  U.  T) 


h(t 


<x)  f  ^  (a)  (o’  +  o)  do- 


(2-5) 
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Here  h  (t)  is  the  impulse  response  function  of  the  filter  and  is  sometimes  variously  called  the 

24 

integrating  function,  scanning  function,  or  window  function. 

The  use  of  correlation  devices  operating  as  practical  correlation  detectors  in  the 
manner  described  mathematically  by  Eq  (2-5)  has  brought  about  the  adoption  of  detection  cri¬ 
teria  based  on  correlation  outputs  in  their  own  right.  An  advantage  of  such  correlation  devices 
is  that  they  are  not  subject  to  an  important  limitation  on  a  device  computing  the  mean  square 
difference.  The  latter  device  must  know  the  precise  value  of  the  component  of  received  signal 
power  due  to  the  transmitted  waveform  as  well  as  the  waveform  shapes  themselves  However, 
the  former  device  need  know  the  waveforms  only  within  the  freedom  allowed  by  an  arbitrary 
constant  multiplier. 

In  reference  to  Eq.(2-2),  it  is  apparent  that  if  the  integrals 

rT  2 

l  V'1  J‘ 

(proportional  to  the  signal  energy)  are  not  equal  for  all  k,  the  criterion  of  maximum  short-time 
correlation  is  not  equivalent  to  that  of  minimum  mean  square  difference.  To  illustrate  this,  the 
geometrical  model  is  consulted. 

The  additive  properties  of  the  vector  representations  of  the  time  functions  were 
shown  to  follow  from  linear  superposition.  The  orthogonality  of  the  components  leads  to  equiv¬ 
alent  vector  interpretations  of  the  average  product  or  correlation  coefficient.  For  example, 
dealing  only  with  the  cosine  terms  of  the  representation  given  in  Eq.(l-5), 

,  rT  TW  X.  7  .  Z.  ? 

X  (t)  Z  (t)  dt  =  -Tf  /  E  - —  cos  ~r- 1  E  -  cos  t  dt  +  (other  terms),  (2-5) 

1  Jo  i=  1  vTW  1  \Tfw  1 


TW  TW  X-Z, 
S  Z  2  1 


TW 


(  1  rT  2iri  2t rk  ,  1 

<  jp  /  cos-T^rt  cos  — tdt 


i=  1  k=  1 

when  the  order  of  integration  and  summation  is  interchanged. 


+  ( 


But 


(2- 5a) 


cos* 


2iri 


,  2trk 

t  cos  —  dt 


J_ck 

y6i 


(2-6) 


k  Ik/ 

where  6^  is  the  Kronecker  delta  and  5.  =  1:  6^  ~  0,  i  ^  k. 

Also,  the  same  behavior  governs  the  sine  terms,  and  all  the  cross  terms  are  zero.  Finally. 


T 


X(t)Z(t)dt 


TW  X.Z. 

E  —  1 
i=  1 


2TW 


(2-7) 


(The  summation  on  the  right  is  obtained  approximately  when  the  alternate  representation  of  the 
symbol  waveforms  in  terms  of  2TW  time  samples  is  employed.  ) 

The  $X.Z.  is  obviously  the  form  of  the  dot  or  scalar  product  of  the  vectors  X  and 
Z.  It  is  related  to  the  distance  between  X  and  Z  by 


z  =  I  |z|2 


1  + 


|X, 


Izp 


(2-8) 
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where  D,  is  used  to  denote  the  distance  corresponding  to  the  k**1  vector-  Again,  it  is  evident 
k  2 

that  if  all  |Xj  are  equal,  the  minimum  distance  corresponds  to  the  maximum  dot  product.  As 

is  shown  in  Appendix  II,  however,  the  magnitudes  of  the  vectors  are  not  all  equal  when  they 

are  segments  taken  from  a  random  Gaussian  noise  waveform.  Where  the  signal  power  S  is  large 

compared  to  the  noise  power  N,  a  case  such  as  that  shown  in  Fig.  2.1  always  has  finite  procabil- 

ity.  Here,  since  |z|  is  constant  for  all  k,  the  correlation 

is  that  constant  times  the  projection  of  the  vector  onto 

the  vector  Z.  It  is  immediately  evident  that  a  vector  might 

occur  with  magnitude  sufficient  that,  although  the  distance 

— ►  — *  -*• 

D^>Dq.  the  projection  of  X^  will  exceed  that  of  on  Z 

When  the  ratio  of  signal  power  S  to  noise  power 
N  becomes  quite  small,  the  term  |X^/Z|  itself  becomes 
small.  Then  the  contribution  of  the  fluctuations  of  the  mag¬ 
nitude  of  to  the  values  of  the  dot  products  is  to  the  value 
of  X^  '  Z  approximately  as  the  signal  amplitude  is  to  the 
noise  amplitude.  Thus,  for  small  signal- to-noise  ratios, 
the  equivalence  of  the  criteria  of  maximum  dot  product  and 

minimum  distance  is  again  approached,  even  though  the  energies  of  each  X,  are  not  equal.  As 

k  2 

is  seen  later  (for  all  S/N),  as  the  system  dimension  n  is  increased,  the  fluctuations  of  |X^| 
decrease  percentagewise,  further  establishing  the  equivalence  of  these  two  criteria. 

The  fact  that  the  criterion  of  maximum  correlation  is  an  optimum  one  for  smalt 
signal-to-noise  ratios,  p  =  S/N,  is  not  of  mere  academic  interest-  It  is  when  p  is  small  that 
the  communication  of  information  becomes  most  difficult.  Here  the  channel  capacities  for  chan¬ 
nels  of  bandwidth  conventionally  associated  with  communications  become  low  enough  that  the  in¬ 
formation  rates  of  even  relatively  low-rate  systems,  such  as  telegraph  and  teletype,  become  a 
significant  fraction  of  the  channel  capacity. 

Furthermore,  there  are  two  features  of  military  significance  that  are  inherently  re¬ 
lated  to  low  signal-to-noise  ratios- 

One  of  these  is  the  possibility  of  communicating  with  received  signal  levels  below  the 
receiver  and  antenna  noise  at  the  receiver  location.  In  the  past,  other  systems  have  been  pro¬ 
posed  which  provide  communication,  although  the  average  signal  power  is  less  than  the  noise 
power  at  the  receiver  input.  However,  these  systems,  such  as  pulse-position  or  pulse-code 
systems,  feature  bursts  of  power  for  relatively  short  periods  of  time  which  are  above  the  noise 
level.  The  average  power  is  less  than  the  noise  power  by  virtue  of  its  being  averaged  over 
larger  periods  of  time.  In  the  proposed  systems,  the  signal  power  can  be  less  than  the  noise 
power  at  all  times.  If  unfriendly  search  receivers  are  limited  to  a  comparable  received  power, 
it  is  highly  unlikely  that  such  receivers  will  be  aware  of  the  presence  of  the  signal  11  on  the  air,  ,r 
unless,  of  course,  the  unfriendly  receiver  performs  the  same  operation  of  correlation  as  the 
friendly  one. 

Another  promising  feature  deals  with  resistance  to  jamming.  As  Fano  has  shown,  7 
the  signal-to-noise  ratio  at  the  output  of  a  correlation  detector  is  given  ideally  by  the  ratio  of 
total  signal  energy  to  noise  power  per  cycle.  This  suggests  the  jamming  power  may  be  forced 


Fig. 2.1.  Vector  mag¬ 
nitude  contributions  of  error. 
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down  to  a  low  value  compared  with  the  signal  energy  merely  by  spreading  the  signal  energy  over 
a  sufficiently  wide  bandwidth.  This  spread  causes  the  jamming  power  to  be  effectively  averaged 
over  the  bandwidth  in  such  a  way  that  the  jamming  power  per  cycle  is  small- 

In  the  next  chapter,  a  specific  description  of  the  systems  suggested  by  the  properties 
of  this  type  of  noise  communication  and  correlation  is  accompanied  by  a  more  complete  discus¬ 
sion  of  what  services  these  systems  are  expected  to  perform  and  their  advantage  in  performing 
them. 
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CHAPTER  III 

DESCRIPTION  OF  THE  FUNDAMENTAL,  NOMAC  SYSTEM 

In  view  of  the  advantages  that  schemes  of  communication  using  noise -like  signals 
and  correlation  detectors  appear  to  possess,  an  extensive  investigation  into  their  properties  has 
been  conducted.  The  code  word  NOMAC  (coined  from  Noise  Modulation  and  Correlation)  has 
been  suggested  for  use  in  referring  to  such  systems. 

NOMAC  systems,  generally  speaking,  trade  bandwidth  for  the  ability  to  operate  at 
low  signal-to-noise  ratios.  In  view  of  the  present  day  emphasis  on  conservation  of  bandwidth, 
these  systems  should  not  find  application  where  interference  is  slight  or  negligible.  However, 
in  certain  military  applications  the  possibility  of  maintaining  communications  secure  from  in¬ 
tercept  or  reliable  in  the  face  of  enemy  jamming  offsets  any  disadvantage  that  may  be  connected 
with  the  use  of  wide  bandwidths. 

The  block  diagram  of  a  fundamental  NOMAC  system  is  shown  in  Fig.  3.1.  The  set  of 
possible  symbols  which  take  the  form  of  finite  duration  segments  of  Gaussian  noise  is  shown  at 
the  transmitter.  The  transmitter  selects  one  of  these  waveforms  X^lt)  and  propagates  it 
through  the  channel  in  which  the  Gaussian  interfering  noise  Y  (t)  is  added.  At  the  receiver, 
copies  of  the  K  +  1  waveforms  are  available  to  use  in  K  •*-  1  correlation  detectors  in  which  the 
received  signal  is  compared  with  each  of  the  copies  representing  possible  symbols. 


TRANSMITTER  CHANNEL  CORRELATION  DETECTOR 


Fig.  3.1.  Block  diagram  of  typical  NOMAC  system- 
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A  problem  of  major  importance  in  the  design  of  NOMAC  systems  is  that  of  deliver¬ 
ing  the  copies  of  the  possible  waveforms  to  the  receiver  so  that  correlation  may  be  performed. 
The  attempts  at  a  solution  of  this  problem  divide  NOMAC  systems  into  two  categories,  one  in 
which  the  possible  symbol  waveforms  are  stored  at  the  receiver,  and  another  in  which  the  wave¬ 
forms  reach  the  receiver  as  reference  signals  through  one  or  more  auxiliary  channels. 

The  first  category,  called  the  stored-signai  system*  presents  rather  severe  require¬ 
ments  of  time  synchronization,  in  order  that  the  correlation  coefficient  calculated  will  corre¬ 
spond  to  the  t  =  o  point  of  the  autocorrelation  function.  One  advantage  of  this  system  is  that, 
while  the  segments  may  be  chosen  at  random  from  white  Gaussian  noise,  they  become  a  known 
set  once  the  choice  is  made.  Thus,  a  scale  factor  may  be  employed  to  make  each  of  the  symbol 
energies  equal,  which  establishes  the  exact  equivalence  of  the  criteria  of  maximum  correlation 
and  of  minimum  mean  square  difference. 

In  the  second  category,  the  synchronization  problem  is  largely  eliminated,  but  noise 
is  generally  present  in  the  auxiliary  channels  also.  Here  the  signals  may  be  randomly  selected 
from  one  or  more  noise  sources  that  are  currently  generating  the  noise.  Obviously,  these  sym¬ 
bol  waveforms  will  be  random  in  all  respects. 

In  Fig.  3.2,  curves  of  the  signal-to-noise  ratio  in  the  output  of  an  ideal  correlation 
detector  as  a  function  of  the  input  signal-to-noise  ratio  and  n  -  2TW  are  shown  for  the  two  cat¬ 
egories  of  NOMAC  system.  It  will  be  shown  subsequently  that,  when  filtering  is  used  for  the  in¬ 
tegration  in  the  correlation  process,  n  is  in  reality  the  ratio  of  the  input -signal  bandwidth  W 
to  the  noise  bandwidth  of  the  integrating  filter.  In  the  figure,  the  signal -to-noise  ratio  p  ,  or 
(S/N)c:,  is  assumed  the  same  for  both  Intelligence  and  auxiliary  channel  or  channels.  From  the 
figure,  if  a  required  output  ratio  and  n  are  known,  one  may  determine  the  permissible  input 
signal-to-noise  ratio.  Conversely,  if  a  desired  input  signal -to-noise  ratio  and  required  output 
signal-to-noise  ratio  are  specified,  the  necessary  bandwidth  ratio  is  easily  obtained  for  either 
type  of  system. 


Fig.  3.2.  Signal-to-noise  improvement 
in  correlation  vectors. 


Fig. 3.3.  Secure  region 
of  communication. 
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To  demonstrate  the  secure  communication  properties  of  such  a  system,  suppose 
that  the  NOMAC  system  as  designed  will  operate  satisfactorily  at  an  input  signal-to-noise  ratio 
(S/N)m£n*  while  an  unfriendly  search  receiver  can  detect  the  radiation  only  at  signal-to-noise 
ratios  greater  than  a  larger  value  (S/N)^^.  Then,  as  is  seen  in  Fig.  3.3,  there  is  a  "secure" 
region  or  annular  ring  about  the  transmitter  in  which  communication  might  be  carried  out  by 
friendly  parties  without  the  knowledge  by  unfriendly  forces  in  that  region 

An  inverse  line  of  reasoning  would  demonstrate  that  an  unfriendly  jamming  trans¬ 
mitter  might  be  believed  to  be  effective  against  the  assumed  transmitted  power  as  determined 
by  the  power  received  at  the  jamming  site  to  some  range  Rq  .  On  the  other  hand,  communica¬ 
tion  would  actually  be  maintained  to  the  inner  radius  IT  ,  and  the  "secure  region"  now  becomes 
a  marginal  communication  area  which  might  cor  eeivably  prove  quite  embarrassing  to  the  un¬ 
friendly  forces. 

As  presented  in  Fif  1  the  ''-’udamen*  al  NOMAC  system  is  idealized  to  facilitate 
the  theoretical  investigation.  The  type  oi  decision  circuit  is  not  specified  in  order  to  permit 
some  latitude  in  the  interpretati  >n  an  ^.plication  of  the  figure.  In  the  following  material,  two 
types  of  decision  circuit  are  evaluated  in  detail.  The  first  of  these  is  one  that  establishes  the 
criterion  of  maximum  correlation  in  its  selection  of  which  of  the  symbols  was  transmitted. 
The  second  decision  circuit  (and  the  easiest  to  *  instruct  in  practice)  sets  as  a  criterion  the  ex¬ 
ceeding  of  a  fixed  threshold.  Called  threshold  detection,  it  indicates  a  signal  as  having  been 
transmitted  whenever  the  correlator  output  corresponding  to  that  signal  exceeds  the  threshold 
value  . 

It  is  not  intended  to  convey  the  impression  that  only  systems  designed  precisely  as 
indicated  by  the  block  diagram  are  included  in  the  analysis  presented.  Modifications  of  the  anal¬ 
ysis  given  here  for  discrete  systems,  along  with  the  work  appearing  elsewhere  concerning  the 
improvement  of  sig ual-to-noise  ratio  in  correlation  detectors/  can  be  made  to  extend  the  cover¬ 
age  to  a  varied  c.ass  of  similar  systems.  For  example,  one  might  use  a  single  random-noise 
source  to  obtain  the  different  symbols  merely  by  using  delayed  versions  of  the  initial  noise  for 
the  sources  of  the  currently  chosen  segments.  The  delay  increments  need  only  be  great  enough 
to  correspond  to  values  of  r  of  the  autocorrelation  function  of  the  noise  for  which  the  correla¬ 
tion  function  is  essentially  zero. 

Other  versions  of  NOMAC  systems  may  include  those  in  which  the  noise -like  wave¬ 
form  is  modulated  in  the  same  manner  in  which  a  sinusoidal  carrier  is  modulated  in  convention¬ 
al  communication  systems.  For  example,  the  transmitted  random  waveform  might  be  varied 
in  amplitude,  frequency  band  of  transmission,  or  relative  time  of  transmission.  These  varia¬ 
tions  correspond  to  conventional  AM,  FM  and  PPM,  for  example.  Obviously,  combinations  of 
any  of  these  modulations  are  possible  just  as  with  sinusoidal  carriers.  The  experimental  mod¬ 
el  shown  in  block  diagram  in  Appendix  VII,  Fig.  3  is  essentially  an  amplitude -modulated  version 
of  a  NOMAC  system,  and  its  theoretical  probability-of-error  treatment  corresponds  to  the  sig- 
nal-to-noise  improvement  type  of  analysis  that  would  be  used  for  an  AM  system. 
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CHAPTER  IV 


THE  THEORETICAL  STUDY  OF  PROBABILITY  OF  ERROR 

A .  The  Criterion  of  Maximum  Correlation 

When  the  receiver  of  a  NOMAC  system  makes  its  decision  about  which  of  the  signals 
XK(t)  was  transmitted  -  based  on  the  criterion  of  maximum  correlation  output  -  the  rate  of  mak¬ 
ing  errors  has  been  obtained.  The  result,  which  is  valid  for  large  n,  and  for  signal -to-noise 
ratio  p,  such  that  the  product  np  is  larger  than  about  2,  is  given  by  the  approximation 


P  (error) 


Kerf 
2  V  l  * 


where 


3rf  Q  =  yijT  exp  -jt2]dt 


For  still  larger  values  of  np,  but  only  for  small  signal-to-noise  ratios,  the  expression  agrees 
with  that  of  Rice,  namely, 


1  F  u  * 

P  (error)  ^ — - - exp  -C'T(l-pr) 

ZVVC’T  L  ^  J 


(4-la) 


Here,  p  «  I,  so  that  |np  &  C'T;  and  K  »  1,  so  that  InK  =  InK  +  1  -  H'T. 

These  results  were  obtained  using  the  geometrical  model  in  the  following  manner. 
First,  there  are  K  +  1  independent  message  vectors  which  are  represented  by  X^,  Xj..  .  , 
Appendix  II  derives  the  probability  density  distribution  for  the  magnitude  of  these  vectors,  a 
type  familiar  in  the  study  of  statistics  For  the  message  vector  2?,  where  S  is  the  average 
signal  power,  it  is  given  as  follows: 


p0(I*) 


(2S)n'2r(-|) 


[  X2] 

-I5-J  ' 


As  is  shown  in  Appendix  II,  this  distribution  has  an  average  value  of 

re-0—) 

^  r~- 5  r* 


and  a  variance  of  not  more  than  l/2S.  The  average  value  is  \l  nS  within  an  error  of  one  part  in 
each  4n  parts. 

Here  it  must  be  pointed  out  that  approximations  are  made  throughout  the  paper,  pri¬ 
marily  because  the  representation  of  the  actual  waveforms  as  having  a  Gaussian  probability 
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density  distribution  is  only  approximately  correct,  and  it  may  be  considerably  in  error  along 
the  "skirts T1  of  the  distribution  curve.  Except  where  otherwise  noted,  linear  systems  are  as¬ 
sumed,  which  is  net  necessarily  the  case  in  practice  and  is  certainly  not  true  over  the  whole 
range  of  amplitude  values  for  which  the  Gaussian  distribution  is  defined.  Since  all  the  results 
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obtained  depend  on  these  two  assumptions  (and  other  assumptions),  results  may  in  some  cases 
be  true  only  to  the  order  of  magnitude  represented-  However,  the  primary  purpose,  which  is 
that  of  indicating  the  behavior  one  might  expect  in  a  physical  system  and  of  demonstrating  how 
the  behavior  varies  with  important  parameters,  is  served- 

In  Fig.  4.1,  the  combination  of  one  of  the  message  vectors,  say,  X^,  and  the  inter¬ 
fering  noise  Y  are  shown  resulting  in  the  vector  Z.  Also,  another  typical  message  vector  5?^ 
is  shown.  The  points  X  lie  approximately  on  a  hypersphere  of  radius  V  nS,  while  Z  lies  approx¬ 
imately  on  a  hypersphere  of  radius  \l  nP-  Here  P  is  the  total  power  S  +  N  where  S  is  the  sig¬ 
nal  power  and  N  is  the  interfering  noise  power.  The  NOMAC  receiver  cross -correlates  the 
received  signal,  vector  Z,  and  each  of  the  message  waveforms,  vectors  X^  As  stated  before, 
this  is  equivalent  to  taking  the  dot  product  between  the  Z  vector  and  each  of  the  set  of  message 

1/n.  The  resulting  set  of  dot  products  is  designated  by  . 

Up  to  this  point,  the  analysis  of  a  NOMAC  sys¬ 
tem  does  not  depend  on  how  the  receiver  makes  use  of  the 
outputs  W^.  However,  for  further  study  it  is  necessary  to 
distinguish  between  the  two  cases:  that  in  which  the  receiver 
decision  circuit  indicates  the  largest  as  the  one  determin¬ 
ing  the  transmitted  signal;  and  that  in  which  a  that  exceeds 
a  previously  established  threshold  value  is  the  indication  of 
the  transmitted  signal. 

For  the  case  in  which  the  criterion  of  detection 
is  that  of  maximum  correlator  output,  it  is  evident  that  no 
error  will  occur  if  the  dot  product  WQ  exceeds  all  other  W^. 
This  in  turn  is  the  case  if  the  component  of  the  vector  Xq 
along  Z  is  greater  than  the  Z -components  of  all  other  vec¬ 
tors  X^.  Since  the  coordinates  of  the  vectors  are  independent  random  values,  the  vectors  them¬ 
selves  are  randomly  distributed  in  n- space.  Appendix  IV  illustrates  this  point  and  shows  that 
the  Z- component  of  any  of  the  K  vectors  not  selected  at  the  transmitter  is  chosen  from  the  same 
density  distribution  as  are  the  coordinates  of  the  message  vectors.  Therefore,  the  probability 
density  distribution  of  the  Z-components  is  given  by  the  Gaussian  function 


vectors  aside  from  the  constant 


Fig.  4.1.  Vector  model 
of  messages,  noise,  and  re¬ 
ceived  signal  (Fig,  1.2), 


P(X‘) 


1 

-  ■  exp 
n/TttS 


(4-3) 


The  prime  is  here  used  to  indicate  the  component  in  the  direction  of  Z. 

The  probability  that  any  one  Z-component  will  be  less  than  the  Z -component  of  XQ 
(hereafter  designated  Xz)  is  given  by 


P(X'<XJ 


-L 


ptX'JdX' 


r00 

1-/  p  (X1)  dX' 

dx 


The  probability  that  all  K 


(4-4) 


(4  -4a) 

vectors  not  selected  for  transmission  will  have 
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Z -components  less  than  X  is  given  by 
P{allX'<X)=  1- 

The  probability  of  no  error  is  the  average  of  the  probability  given  over  all  values  assumed  by 
X^.  This  is  indicated  formally  as 


X' 


1  iS- 


p(X')  dX' 


(4-5) 


f” 

P  (no  error)  =1  p  ,(X  )  P  (all  X'<Xz)  dX2 


(4-6) 


from  which 


P  (error)  =  1  -  f  ”  p,(X  )  R  -  f  P(X')dX'l  dXz 

J  -oo  L  Jx  J 


(4-7) 


The  function  p1(Xz),  the  probability  density  distribution  function  of  Z -components 
of  the  vector  Xq*  must  be  determined  in  order  to  carry  out  the  indicated  integration.  This  is 
one  of  the  more  difficult  aspects  of  the  problem,  and  a  considerable  portion  of  the  total  research 
has  been  devoted  to  finding  the  properties  of  and  suitable  approximations  for  the  density  distribution. 


B.  The  Density  Distribution  of  Xz 

As  shown  in  Fig.  4.2,  the  vector  7  can  be  broken  into  two  components,  one  of  which 
is  along  and  one  normal  to  XQ.  Furthermore,  these  two  components  are  independent.  This 


Fig.  4.2.  Message,  noise,  and  signal 
vectors  showing  components  of  vectors. 

:zi  =  *[{  |xo:  +  y2)2  +  Y*  . 


conclusion  is  reached  after  noting  that  the  n  com¬ 
ponents  making  up  vector  ¥  are  independent, 
and,  according  to  the  methods  of  Appendix  IV, 
may  be  thought  of  as  independent  after  rotation 
of  the  coordinate  axes,  since  7  is  independent 
of  the  vector  onto  which  the  initial  axis  is 
aligned  in  this  example.  There  are  thus  three 
independent  quantities,  ,X0I ,  Y^  and  Y^»  that 
combine  to  yield  vector  £  in  a  right -triangle 
relationship.  As  expressed,  Yj  is  always  pos¬ 
itive  (a  magnitude)  while  Y^  may  be  positive  or 
negative. 

(4-8) 


A  knowledge  of  the  right  triangle  formed  by  these  components  leads  to  a  fairly  simple  expres¬ 
sion  for  X  in  terms  of  these  quantities,  namely, 
z 


x„  =  jx. 


^  +  YE 


I  +  y/  +  Yf 


(4-9) 
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A  somewhat  more  easily  handled  expression  is 


(4-10) 


(IXJ  +  Y2)2 


!x  r 

x  =  ±  - 

z  / 

1 / ,  .  Y1 

M  1  T  - - T 

V  <''Xo'+Y2> 


xz<o  ifY2<-  |XJ 


X  >  0  if  Y,>-  |x  I 

2  l  1  O  1 


(4-11) 


Then,  by  steps, 


tv; 


-<a  ,  a  <  0;  (for  Y^  <  — 


(ixj  +  Y  Y 


(4-12) 


lXol  +  Y2>Z 


<  a  ,  a  >  0;  (for  Y2  >  -  |Xo|) 


(4 -12a) 


With  due  regard  for  the  sign  of  a,  an  intermediate  step  follows,  valid  for  both  Eq.(4-12)  and 
Eq.(4-  1 2a) . 


p[xz<a]sPi|Xj  ,Y2);<  '  2 

L  lxo  I  r a 


(4-  12b) 


P[xz<a]  =  p|(|Xo;  ♦  Y2)  <  4  y 


for  Y,>  -  |X  | 
Z  1  o' 


(4-13) 


i,lxJ,vp<l  iT^J' 


for  Y->  <  -  |X 
Z  'o' 


(4*- 13a) 


TXJ  ’v<aV^? 


by  sign  of  a, for  |a|  <;  |X  j  ,  (4  -  13b) 
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-a2'^  •  f0r|“U 


(4-14) 


Obviously,  |X  |  <|X  |  ,  and  therefore  P  (X  <a),  is  unity  for  all  a>]X  |  ,  while  :t  is  zero  if 
a<-|X0l- 

Expression  (4-14)  is  derived  for  fixed  |Xq;  and  Y  ^  -  The  average  probability  that 
Xz  is  less  than  a  is  obtained  by  averaging  over  ail  |X^  and  Y^. 


P.[X  <a]  =  f  p(ixi)d:x;r  p  (Y.)dY.  P  Y,<a,/ - 

1  z  Jo  °  0  '  0  Jo  ^  1  L  V  |XQ|  - 


-5  -  X 
2  f  o 


(4-15) 


in  which 


P  Y^<aW - p  -  X 

2  Ki  -  ‘  ° 


for  a  <  —  |x  ! 

1  o 


r  \f  |X 

I 

'-'-00 


L.'x, 

^  o' 


!  |Xo|“— a 

p(Y,)dYa 


aY. 

1  -  IX  . 

/,„  1 2  2  0  , 
v  |X  |  -a  vz 


for  -  |X  I  <  a  <  |X  ! 

'O’  o 1 

for  a  >  IX  | 


(4-16) 


A  sketch  of  cumulative  probability  distribution  P[X  <  a]  for  particular  |X  jand  Y, 


is  shown  in  Fig.  4.3. 


Fig. 4.3.  Probability  distri¬ 
bution  of  components. 


War  |Xp| 
|X0|e+Yi2 


With  the  aid  of  Fig.  4.3,  it  is  seen  that,  when  a  is  positive, 


aY. 

l!g 


r»a  p  OO  pec  n  ij  |a  |  —a 

P(Xz<a ]  =  /  P0(lXo  )d;Xo'  T  /  Po(lXol)dlXo'  /  Pa(Yl)dYl  /  P(Y2,dY 

°  o  •'a  ^  o  ^-co 


(4-17) 


for  a  <  o. 
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aY. 


P[Xl<‘*’  *i|a”  Po(1Xol)d'!!olX”|,qIYl)dYl/„ 


Vixj^ 


T  - ! x„l 

p(Y;)dY2 


The  probability  density  distribution  function  is  obtained  by  differentiating  Eq.(4-17) 
and  Eq.{4-17a)  with  respect  to  a  (see  Ref.  8 ,  page  55). 


aY, 


pi(a,=  i>o<ix  iwix0'  */%„<:=y>di>yjf  VW  AfJ 


HXi 


O’ 


p(Y2)  <MZ 


aY, 


»o‘lXol>X”  £ 


SF? Hxpl 


p(Y,)dY2 


|Xj=a 


or 


P!(a)  =  P0(|X0|)d  |X0|  Pq(Yj)  dYj  p 


aY, 


-l/l  Xo|2  "£ 


(4-18) 


(4-1 8a) 


which  becomes 


p,(.) 

When  a  is  negative, 


(ix/^)*2 


aY, 


T7  -py 


aY. 


reo  .oo  niix  \Z~/  _|X°! 

PI(a)Va  °  P(Y2)dY 


(4- 18b) 


aY 


-lx. 


xT-a2  "'° 

P(Y2)dY2| 


iX„|=-a 


(4-19) 


or 


piui  ■  s~  po«xoi>di  x„i  r"  pqw,  my,  — !5L^_  p 

**  •/  o  1  /  \C  c * 


(|X0|2-a2) 


aY 


L--|x  1 

— r  1  rJ 


1 


(4- 19a) 


22 

CONFIDENTIAL 


CONFIDENTIAL 


When  written  in  terms  of  X  ,  the  desired  probability  density  distribution  function 


for  Z-components  of  the  vectors  jXQj  reads 


ppy  ,  Po<ixo»d ixo^X”pq<Y»,<iv 


|xol2v, 


X  Y. 
z  1 


1  3/2 

(]x0i  -X,> 


'J'X  F-xa 


-i*0r 


(4-20) 


IIX,  I 


In  Eq.(4-20),  the  p  without  a  subscript  denotes  a  normal  probability  distribution  function  with 
variance  N,  while  pQ  is  used  to  designate  a  distribution  of  the  type  considered  in  Appendix  II, 
in  this  case  of  dimension  n  derived  from  a  variance  S.  The  other  function  distinguished  by  the 
subscript  q  is  of  the  same  type  as  pQ,  but  of  dimension  n  -  1  and  with  S  replaced  by  N,  the 
noise  power. 

It  is  therefore  apparent  that  in  Eq.(4-20)  is  a  Gaussian  distribution  function 

modified  by  the  ratio 


1  -- 


(1*0I  ~Xz) 


X 


-3/2n 


and  subsequently  averaged  over  all  |Xj  greater  than  Xr/  and  over  all  Y  .  It  is  immediately 
evident  that  the  work  necessary  to  determine  a  precise  expression  for  p^X^  represents  an  un¬ 
wise  investment  in  time,  since  the  precision  of  the  original  assumptions  does  not  justify  the 
labor.  However,  an  examination  of  the  factors  contributing  to  pj(X^)  can  lead  to  a  useful  ap¬ 
proximation. 

First,  since  p^X^)  is  largely  an  averaged  form  of  a  Gaussian  distribution  function, 
it  should  itself  tend  to  a  bell-shaped  distribution,  particularly  for  large  n  for  which  the  aver¬ 
aging  probability  functions  approach  impulse  functions.  It  follows  that  the  average  value  and  the 
variance  of  the  distribution  can  be  used  to  obtain  a  suitable  approximation  under  certain  con¬ 
ditions. 

The  average  value  of  is  difficult  to  obtain,  but  it  approaches  'vAn/p  S  for  large 
n  and  does  not  differ  greatly  from  this  value.  This  fact  is  demonstrated  by  expanding  the  ex¬ 
pression  for  Xz  in  terms  of  |X  [,  Y^,  and  in  a  Taylor  series  about  the  value  of  X  assumed 
when  each  of  these  variables  takes  on  its  average  value.  It  then  can  be  shown  that  the  expecta¬ 
tion  E  (Xz  -  X^)  is  an  order  of  magnitude  smaller  than  |Xj2  which  represents  this  average 
value  squared.  The  first  term  of  the  expansion  is 


ax  =  x„  -  x,  £ 

z  z  z 


3|X 


ax„ 


a|xq|  ♦ 


BY 


AYr  + 


9Y. 


AY- 


(4-21) 


Since  the  standard  deviations  of  I  X 


1  ~*2 

Yj  and  are  small  compared  to  the  vectors  making  up 


the  triangle  (for  large  n),  the  first  term  is  used  to  represent  all  significant  contributions  to  X  . 
From  their  independence,  the  variance  of  the  sum  is  the  sum  of  the  variances  of  the  components, 
namely, 

2 


X 


r,x« 

- 

2 

'axz 

2 

faxz  ] 

[•iV 

X,  <r  'X  , 
z  ■  o' 

+ 

_9Y1 

XZ 

+ 

9Y2  Xz  <ry2 

(4-22) 
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sx,  [wy 
W 


(4-23) 


8Xz  .  [<l-i  *  •  vf]iX0l  -  (|Xo;  *  Y2)2ixj 


av. 


[(|Xo!  +  y/  -  vf] 


V'2 


(4-24) 


9XZ  !Xo'YI  f!Xo|  +  Ya) 

8Y2  >  i  >p/2 

ki‘v2^vf] 


(4-25) 


When  the  partials  are  evaluated  at  the  mean  Xz  and  inserted  in  Eq.(4-22)  the 


result  is 


s2[s  +  (2  -“)Np  +  2SN3(1  ~^)z  +  S2N2(1  - -^) 
2(S  -  N)3 


(4-26) 


Since  large  n  has  already  been  postulated,  neglect  of  the  terms  l/n  puts  Eq.(4-26)  into  the  form 


N 


(4  -  26a) 


The  validity  of  Eq.(4-26a)  depends  upon  the  smallness  of  the  ratio  /Xz  ,  which 
is  of  the  order  V S  -f  n/P  S.  Thus,  if  nS/P»l,  Eq.(4-26a)  is  useful-  The  earlier  stipulation 
of  large  n  is  consistent  with  this  requirement.  It  is  noted  that,  for  small  signal-io-noise  ratios, 
nS/P  =  nS/N.  This  in  turn  is 


nS  _  2TWS  _  fs 
N  N  ‘  N 

o 


-np 


(4-27) 


Here  E^  is  the  total  signal  energy,  the  noise  power  per  cycle.  Here  again,  p  is  used  to 
indicate  the  signal-to-noise  ratio  in  the  signal  or  intelligence  channel  and  2(E and  np  are 
used  interchangeably,  the  latter  being  preferred  when  brevity  is  an  asset. 

The  average  value  and  the  variance  lead  to  the  writing  of  an  approximation  for 
pj  (Xz)  in  Gaussian  form,  which  is  valid  for  large  n  as  follows: 


p’^,i7^‘xp 


(-  (xz-s 


(4-28) 


This  expression  is  reasonably  close  to  the  true  distribution  over  the  bell-shaped 
part  of  the  curve,  which  is  the  significant  portion  of  the  distribution.  Fortunately,  while  Eq.(4-28) 
may  be  considerably  in  error  along  the  skirts,  its  role  in  the  integration  is  only  one  of  aver¬ 
aging,  and  the  parts  most  in  error  contribute  least  to  the  final  result.  The  related  distribution 
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of  the  normalized  value  X  ~  S  \/"n/p'  is 

z  1  * 


'i 


7ifs> 


J  -  /S  ~ 

^  "  (  nS  ' 


exp 


nS 


(S  +  2N) 


1  - 


(4  -28a) 


When  this  substitution  is  properly  made  in  Eq.  (4-7),  the  result  obtained  ;s 


P  (error) 


exp[  S  t  2N  X*] 


W  J  (S  "  2N) 
*  T  — nS — 


T 


exp 


[-v] 


|nS 

Vp 


x 


V  2  tt 


K 


dX 


(4-29) 


It  is  observed  that  the  distribution  p.(X  )  has  an  essentially  constant  spread,  but  the  average 

i  z  ^ 

value  of  Xz  increases  as  the  square  root  of  n.  The  spread  or  variance,  <r^z,  goes  to  zero 
with  the  signal  power,  goes  to  1/2 S  as  the  noise  power  approaches  zero,  and  never  exceeds  the 
signal  power.  Thus,  the  variance  becomes  small,  percentagewise,  with  increasing  n  .  The 
normalized  distribution  function  approaches  an  impulse  function. 

In  view  of  these  properties,  one  will  note  that  the  probability  of  error  given  by 
Eq.(4-8)  will  not  differ  much  from  the  result  acquired  by  substituting  the  average  in  the  ex¬ 
pression  in  brackets,  rather  than  performing  the  indicated  averaging-  Then, 


P  (error)  ^ 

or,  by  change  of  variable. 


p  (X)  dX 


(4-30) 


P  (error)  ~ 


(4 -3  0a) 


If  the  integral  is  small  compared  to  unity  (note  that  if  v  nS/p  is  about  1.3,  the  integral  is  less 
than  0-1),  a  further  approximation  is  given  by 


P  (error) ^ 


(4 -30b) 


For  values  of  v  nS/P  consistent  with  Eq.  (4-26),  one  may  use  the  further  approximation 

/Q  e*p[-t2/2] dt  =  ^e*p[-^2/2] 


which,  when  used  here,  results  in  the  expression 

P  (error)  ^  K  (^)  exp  [i-Bpj  .  (4-31) 

A  comparison  of  Eq.(4-30)  with  the  expression  of  Eq.(A3-6)  shows  the  expected  simi¬ 
larity.  Therefore,  here  too,  if  the  approximations  nS/2P  =  C'T  for  S  «  N,  and  K  =  exp[H‘T]  for 
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K  »  1  are  used,  an  expression  for  the  probability  of  error  is 

P  (error)  ^ - -  exp  [-  C  'T(l  -  1  .  (4-31a) 

2Vtr  C*T  L  U  J 

The  implication  of  expressions  of  the  Eq.(4-31a)  is  that  the  probability  of  error,  and 
thus  the  per-unit  equivocation,  can  be  held  arbitrarily  low  by  increase  of  T  (involving  delay)  if 
only  the  H/C  ratio  is  less  than  unity  by  an  amount  however  small.  However,  this  expression  is 
valid  only  for  small  signal-to-noise  ratios.  For  larger  values  of  S/N*  the  ratio  S/P  approaches 
unity  and  is,  of  course,  smaller  than  ln(l  +  S/N).  It  follows  that  H  (for  large  K)  must  be  less 
than  a  number  smaller  than  the  theoretical  capacity  if  the  exponential  [in  K  -  nS/2P]  is  to  remain 
negative . 

It  is  of  incidental  interest  to  note  that  Shannon  shows  (see  Ref.  22,  page  63  —  "White 
Gaussian  noise  has  the  peculiar  property  .  .  .  ")  that  any  random  waveform  can  be  used  in  a  sys¬ 
tem  disturbed  by  white  Gaussian  noise  and  can  approach  the  ideal  when  the  signal-to-noise  ratios 
are  small.  This  fact  is  illustrated  in  Golay's  result^  for  pulse  position  modulation  under  the 
assumption  of  small  signal-to-noise  ratio.  It  is  worth  repeating  that  it  is  the  distinct  advantage 
of  correlation  techniques  that  they  present  the  most  suitable  methods  of  accomplishing  the  prac¬ 
tical  identification  of  randomly  varying  signals  buried  in  noise. 
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CHAPTER  V 


THE  THEORETICAL  STUDY  OF  THRESHOLD  DETECTION 
A.  The  Probability  of  Error  for  an  Arbitrary  Threshold 

Threshold  detection  is  given  the  distinction  of  a  separate  chapter  because  it  repre¬ 
sents  a  technique  in  common  use,  and  thus  deserves  an  accurate  and  complete  analysis  for  com¬ 
parison  with  the  foregoing  more-or-less  ideal  methods  of  reaching  a  decision  about  the  trans¬ 
mitted  signal. 

In  review,  a  threshold-detector  scheme  is  one  in  which  the  decision  about  which  of 
the  possible  message  waveforms  was  transmitted  is  based  on  one  of  the  outputs  (see  Fig.  -LI) 
exceeding  a  predetermined  fixed  value  called  the  threshold.  Of  course,  if  two  or  more  outputs 
exceed  the  threshold  value,  or  if  none  exceeds  it,  no  decision  can  be  made,  and  an  error  results. 
Obviously,  there  is  also  an  error  if  only  one  of  the  outputs  exceeds  the  threshold  but  if  that  output 
does  not  correspond  to  the  transmitted  message. 

A  first-order  expression  of  the  effect  of  establishing  an  arbitrary  threshold  can  be 
outlined  as  follows.  In  Fig.  5.1,  for  a  given  received  signal  Z,  the  correlation  output  is  the 

segment  RO  times  |z| .  It  is  assumed  that  an  arbitrary 

P 

fraction  of  the  expected  output  is  chosen  as  the  threshold, 
and  that  OQ  is  the  corresponding  component  along  Z  neces¬ 
sary  to  produce  the  threshold  value  in  the  output  of  a  corre¬ 
lator.  The  expected  output  is  nS,  and  the  threshold  is  anS, 
where  a  is  the  "threshold  coefficient.  11  Thus,  the  segment 
OQ  corresponds  to  aS V n/F. 

The  possibility  that  a  combination  of  X^  and  Y 
might  occur  to  yield  a  Z-component  of  less  than  OQ  is 
w  neglected  for  the  moment,  and  only  the  probability  of  error 

caused  by  the  occasional  occurrence  of  a  component  of  an 

...  Fig.  5.1.  Vector  model  xt  ^  ^  Q)  in  the  Z-direction  which  exceeds  OQ  will  be  in- 

illustrating  threshold  detection.  k  x  ^  H 

vestigated. 

The  vectors  X^  are  randomly  oriented  in  n-space, 
and  to  a  first  approximation  are  of  equal  length.  Thus,  they  terminate  randomly  on  a  hypersphere 
that  is  the  locus  of  all  points  of  distance  |x|  =  J'fnS  from  the  origin.  If  one  of  the  vectors  X^ 
should  terminate  in  the  zone  above  Q  in  Fig.  5.1,  its  Z-component  will  exceed  OQ  and  an  error 
will  occur.  The  probability  of  errors  due  to  this  cause  is  thus  approximately  equal  to  the  ratio  of 
the  area  of  the  zone  to  the  area  of  the  sphere  for  any  one  random  vector.  The  area  of  the  zone 
will  not  exceed  the  area  of  a  hemisphere  of  radius  h  (see  Fig.  4.1).  Thus, 


P(Xk  lies  in  zone)^-^^ 


Here  A  (h)  is  the  area  of  a  hypersphere  of  n  dimensions  as  discussed  in  Appendix  II  (see  All- 17) 
and  with  radius  h.  From  Eq.  (5-1),  it  follows  that 
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,n- 1 

P(X,  lies  outside  zone)  :>  1 - r 

k  2|X,  n" 1 


(5-2) 


!  r  2,  z  ~\ 

But  h  is  seen  to  be  y  1  —  l^(a  nS  )/P|  »  while  |x|  is  the  radius  of  the  hypersphere, 
ynS.  From  these  values,  Eq.  (5-2)  may  be  rewritten 


P  (X^  lies  outside  zone)  >,  \  — ~  ^1  — 


n-  1 


(^3) 


>1  IfN  +  SJl-a 
^  2 


n~  1 

2v\  2 


N  +  S 

Since  K  is  always  one  or  greater,  one  may  readily  write 

n- 11 

2  S1 


P(all  X^  lie  outside  zone)  ^ 


1  —  a 


K 


(5- 3a) 


(5-4) 


and  from  Eq.(5-4)  the  probability  of  error  from  this  cause  (components  of  wrong  vector  s  exceed¬ 
ing  OQ)  is 


P  (error)  ^  1  - 


K  +  1 


n-  r 

«-*( 

l-a2S)  2 

1  a  p) 

Hi) 


n-1 

2 


(5-5) 


(5-5a) 


The  validity  of  Eq.  (5 -5a)  is  insured  by  subtracting  the  smaller  value 

n-J. 

,  K  /.  2Sp 

from  unity  rather  than  the  larger  indicated 


•if-'*) 


n-1' 

2 


K 


Also,  K  +  1  is  certainly  larger  than  the  K  for  which  it  is  substituted-  From  Eq.  (5-5a)  it  follows 
that 


P  (error)  < 


K  +  1 


h  l~ S 


2  Ai  1  —  a 


/  N  \  /  N  +  S(1 
l  N  +  S  M  N 


-^1 


(5-6) 


or 


r  r  - 

log2  (K+l)  ^-p-j  +  ~z  lo§2[l+  N 


a2) 


(5-7) 
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In  terms  of  capacity  and  information  rate  of  the  source,  this  becomes 

r-w  1  -.HT-CT+4J-10E,  i+fr<l-a2) 

P  (error)  < - 2  2  fc2|_  N  J  .  (d-8) 

?  *fi  rm 

2^1—3  p 

In  Eq.  (5-8),  increasing  the  signal  duration  T  will  lead  to  any  desired  frequency  of 

errors,  however  small,  if  CT  exceeds  (r/£)  log^  [^1  +  (S /1ST )  <  1  —  a2)J  +  HT.  This  is  equivalent  to 

subtracting  from  the  theoretical  capacity  a  capacity  equal  to  that  of  a  similar  system  in  which 

the  received  average  power  is  (1  —  a^)  times  the  original  signal  power,  and  requiring  that  the 

information  rate  at  the  source  be  less  than  the  diminished  capacity.  For  small  signal -to- noise 

Z 

ratios,  the  remaining  useful  capacity  is  only  a  times  the  theoretical  capacity.  From  a  differ¬ 
ent  point  of  view,  the  transmitted  power  is  only  a  times  as  effective  as  the  th  optical  limit,, 
and  a  corresponding  increase  in  power  might  be  utilized  to  regain  the  performance  that  would 
correspond  to  the  ideal  use  of  the  original  power. 

In  the  foregoing  manner,  it  is  shown  that  an  arbitrary  choice  of  a  standard  threshold 
will  result  in  a  loss  of  useful  channel  capacity.  However,  the  careful  choice  of  an  optimum 
threshold  will  lead  to  an  efficient  use  of  all  the  system  capacity  if  certain  conditions  are  met,  as 
will  be  shown  subsequently. 

To  obtain  more  precise  results  than  the  upper  limit  expressed  by  Eq.  (5-8),  it  is  nec¬ 
essary  to  include  all  the  conditions  under  which  errors  are  recorded.  A  message  is  correctly 
indicated  when  the  threshold  value  is  exceeded  only  by  the  correct  correlation  output  W  and 
not  by  any  other  W^.  Any  other  combination  will  result  in  an  error. 

Therefore,  the  probability  of  no  error,  given  for  a  fixed  magnitude  of  received  vec- 
tor  Z,  can  be  expressed  as  follows: 

P  (no  error/ |Z|)  =  P(WQ>anS/|z|)  [p(Wk<anS/lz|)]  K  .  (5-9) 

The  probabilities  are  given  as  functions  of  |Z|  because  it  is  necessary  to  compare  all  W  ob¬ 
tained  by  correlating  the  various  possible  signals  with  a  single  received  signal.  The  average 
probability  of  error  is  then,  obtained  as  an  average  of  the  probability  of  error  associated  with 
each  received  combination  of  signal  and  random  noise.  Thus, 
r  00 

P  (no  error)  =  f  pQ(|Z|)  d  |Z|  P  (no  error/ |z| )  .  (5-10) 

u  o 


In  Eq.  (5-9),  the  term  in  brackets  is  found  (in  accordance  with  the  methods  of  Appen¬ 
dix  IV)  to  be  given  by 


P(Wk<anS/|Z|>  =  f 
J  -Q 


exp  — 


h  dwk 


(5-10 


PCW.  <ranS/|z|) 


7  Zv  *  -00 


’  -V ]dv 


(5-1  la) 
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by  change  of  variable. 

The  probability  that  Wq  exceeds  anS  is  not  so  readily  expressed.  However,  a  large 
part  of  the  difficulty  may  be  avoided  as  follows: 


P  (error)  =  1  -  f  p  (|Z|)  d  |Z|  [l  -  P  (W  <anS/iz!)l  1  -  _  P-1 — —  civ 

Jo  °  L  °  J  •'an  4S  'f&T 

L  ~w 


4\ 


2  \  IK 


.  (5-12) 


U) 


joro  noc  exp  y - 

p  (|z|  )d|Z|  P  (W  <anS/ |Z| )  +  K  /  _  - —  dv 

o  “'an  "V’S  >Jztt 

L 


(5- 12a) 


In  Eq.  (5- 12a),  all  terms  involving  products  of  the  probabilities  of  individual  error  are  neglected, 
since  in  practical  cases  each  is  very  small,  and  terms  of  higher  orders  of  smallness  contribute 
little  to  the  true  result.  From  Eq.  (5-12a),  one  may  write 


1  (error)  =  P  (W  <  anS)  +  K  f  °°  p  ( |z| )  d  jzj  f 
°  Jo  J& 


■oo  exp 


U) 


dv 


(5-13) 


The  first  term  on  the  right  in  Eq.  (5-13)  is  the  average  probability  that  W  is  less  than  anS  and 
no  longer  a  function  of  |z| .  A  study  of  the  probability  density  distribution,  p^  (W^),  which  leads 
to  P(WQ<anS),  is  given  in  Appendix  V.  When  it  is  introduced,  the  probability  of  error  becomes 


P  (error)  =  f^p^W  )dW  +  K  f”  (|z|)d  |z|f° 
‘'-oo  Jo  J  ai 


U) 


an  Vs" 

~[zr 


(5-1 3a) 


In  Eq.  (5-13a),  po(]z|)  is  of  the  type  already  frequently  used,  but  stems  from  a  normal  distribu¬ 
tion  with  variance  P  =  N  +  S. 

The  area  of  (WQ)  is  distributed  about  the  average  value  nS,  while  the  standard 
deviation  of  the  distribution  is  VnS  (N  +  2S).  Thus,  for  large  values  of  n  and  for  a  somewhat 
less  than  unity,  the  first  term  on  the  right  of  Eq.  (5- 13a)  will  be  much  less  than  the  second  (to 
approximately  the  extent  that  K  exceeds  one).  Therefore,  the  probability  of  error  behaves  very 
much  like  the  second  term  alone.  Furthermore,  the  final  result  will  not  differ  greatly  from  that 
obtained  by  substituting  the  average  value  of  |z|  in  the  second  integral,  since  this  distribution, 
too,  is  quite  peaked  about  its  average  value  i nP  for  large  values  of  n.  It  follows  that  for  n 
large  and  for  a  in  the  range  0  <  a  <  0.8,  one  may  write 

r00  exp(~x) 

P(error)  ^  K  /  - dv  .  (5-14) 

‘'an  VS"  VZtt 

V  nP 

When  the  approximation  used  in  Eq.  (4-30'  is  applied  here,  there  results 

P  (error)  ~  K  ■  f\J~^  exp  a2  (5-15) 


(5-15) 
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To  gam  still  further  insight  into  the  meaning  of  this  expression  (but  subject  to  the 
conditions  of  K  »  1  and  S  «  Nj,  ether  substitutions  as  used  in  Chapter  IV  lead  to  the  result 


P  (error)  ~  —  1  expf-  a^C'T 

?  ^  1  rr  *-  a  C  _ 


2  1  ira  C'T 


(5.16) 


The  expression,  subject  to  the  same  conditions  discussed  under  the  inequality  (5-8), 
affirms  the  fact  that  the  probability  of  error  may  be  made  arbitrarily  small  with  increasing  T  if 
the  ratio  of  information  generated  at  the  source  to  the  theoretical  system  capacity  does  not  ex¬ 
ceed  the  square  of  the  relative  threshold  coefficient  a.  This  decrease  of  effective  capacity  is 
made  apparent  by  the  appearance  of  a^C  in  each  place  that  C  appears  in  Eq.  (4-31). 

B.  The  Optimum  Threshold 

As  developed  in  the  preceding  section,  the  probability  of  error  when  a  threshold  cri¬ 
terion  of  detection  is  used  can  be  written 

K 


n  oo  r  r 00  nr  r an^ 

’(error)  =  1  -  /  p(|Z|)d|Z|  /  P3(W/|z|)dWo  /  p  (Wk/|z|)dWk 

Jo  L^anS  J 


(5-17) 


The  density  functions  p^  (Wq/|z|)  and  p4(W^/|z|)  have  not  been  used  previously  in  this  paper, 
and  they  represent  merely  the  results  of  differentiating  with  respect  to  the  appropriate  parameter 
of  the  probabilities  P  (WQ>  anS/ |z[ )  and  P  (W^<anS/  |Z| ),  respectively.  To  determine  the  optimum 
value  of  threshold  coefficient  a  which  leads  to  the  minimum  probability  of  error,  P  (error)  is 
differentiated  with  respect  to  a,  and  the  resulting  function  examined  for  zeros. 
c00 

8P  (error) 


aa 


fr  ^anS 

K 

- 

[jT  P4(wk/|z|)dwk 

-  p3(anS) 

=  -[  P0(|Z|)d 

Jo 

Jr*oo  /*  anS 

P3(W  /|Z|)dWo  K  /  p4(Wk/ 
anS  J  [y~oo 


zhdw, 


K-l 


p4 (anS) 


which  is  zero  if 


/anS  /'oe 

P4(Wk/|Z|)dW  =p4(anS)K/  P3(W/|z|)dW  . 

oo  JanS 


-oo  "  ~*anS 

Thus,  a  zero  of  Eq.  (5-18)  and  a  minimum  of  P  (error)  occurs  when 

>oo 

n.  fW  /IzhdV. 

O 


(5-18) 


(5-19) 


p,  (anS)  =  p4  (anS|  •  K— -j- 


r*  anb 

/  P4(W  /|Z|)dW 

J  -oo  ^ 


(5-20) 


Since  p4(W^/|Z|)  is  a  modified  normal  distribution  about  zero  mean,  and  p^(Wo/|Z|) 
is  distributed  about  a  mean  value  nS,  for  all  a  between  zero  and  unity,  the  ratio  of  integrals  in 
Eq.  (5-20)  must  lie  between  1/2  and  2.  When  n  is  large,  the  peaking  of  the  distributions  leads  to 
a  ratio  approaching  unity.  Thus,  for  large  n,  one  may  write 

p3  (anS)  =  Kp4(anS)  .  (5-21) 
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One  may  make  use  of  the  fact  that  the  variance  of  variable  W  is  given  by  nS  (N  +  2S)» 
while  that  of  is  nSN.  To  a  first-order  approximation,  if  the  p's  are  represented  by  normal 

distributions,  Eq.  (5-21)  becomes 


U-n$  (N  +  23) 


(anS  -  nS)  „  1  (anS) 

eXPf2nS(N-+2STj  =  K  ^  I' "  ^ 


(5-22) 


(l-a)2nS  -  1  a2nS 

eXp  -  2  (N  +  2S)  K^eXP  "TlT 


V  K  +  2S  L  J  1 

From  Eq.  (5-22a),  it  is  evident  that 

(I  -  a)2  _  a2  2  (N  +  2S 

n  Tzs,  '  lnK 

The  solutions  of  this  equation  are 


(5-22a) 


(5-22b) 


(5-23) 


When  n  is  very  large  (which,  incidentally,  is  necessary  if  the  normal  approximation  to  the  func¬ 
tions  and  p4  is  to  be  valid)  and  for  S/N  «  1,  the  solution  becomes 

a  =  jalfl  +#+  -Wl  +4lnKt  ♦•)  -  ll  ,  (5-24) 


(5-24a) 


tions 

and 

.  IM 

a  “  2S 

or 

.  ^  N 

When  the  indicated  operations  are  carried  out. 


•H1 


1  +  —  ll 
nP 


(5 -24b) 


Under  the  conditions  of  K»l,  and  those  outlined  before,  such  that  the  approximations 
l/Z  np  =  C'T  and  lnK  =  H'T,  Eq.  (5 -24b)  maybe  written 

a  '  2*  1  +  If]  (5‘25) 

This  expression  has  appeared  in  slightly  different  form  in  the  note  by  Golay,^  and  is 
the  result  obtained  by  all  such  optimizations  of  small  signals  detected  by  the  threshold  criterion 
of  detection.  The  important  point  brought  out  by  this  result  is  that  if  one  is  to  obtain  the  maxi¬ 
mum  use  of  a  given  channel  when  threshold  detection  is  employed  (subject  to  the  conditions  listed 
above  for  the  validity  of  the  result),  he  must  adjust  the  threshold  coefficient  quite  close  to  unity. 
In  fact,  the  difference  between  H/C  and  unity  must  be  approximately  halved  by  the  difference  be¬ 
tween  the  threshold  coefficient  and  unity  for  the  optimum  result.  Equation  (5-25)  also  illustrates 
the  fact  that,  if  a  channel  is  to  be  used  to  carry  information  at  a  rate  substantially  less  than  the 
theoretical  channel  capacity,  there  exists  a  definite  threshold  for  which  the  errors  are  a  mini¬ 
mum. 

When  the  optimum  value  of  a  is  placed  in  Eq.  (5-16),  the  result  is 
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P  (error) 


exp 


C'T  / 

i  S-fl 

~  4  { 

1  c  J  J 

(5-26) 


N  rrC'T  (l  + 

It  is  emphasized  that  expression  (5-26)  is  not  valid  for  very  near  unity  (and  thus  H/C  near  unity) 
but  is  significant,  however,  for  H/C  of  the  order  of  one-half  or  so, 

A  further  study  of  the  effects  introduced  by  the  use  of  a  standard  threshold  for  corre- 
lation  detection  can  be  made  by  postulating  a  highly  idealized  system  of  the  following  type. 

A  set  of  K  +  1  vectors  is  arranged  in  n-space  so  that  they  are  mutually  orthogonal. 
Obviously,  n  must  exceed  K.  The  radius  cf  the  hypersphere  on  which  all  K  +  1  vectors  termi¬ 
nate  will  be  jxj  =  ^nS.  Under  these  conditions,  a  random  orientation  of  the  vector  system  in  a 
set  of  Cartesian  coordinates  of  the  n-space  will  cause  the  coordinate  values  to  be  measured  by 
numbers  which  very  nearly  (but  not  exactly)  follow  a  Gaussian  distribution  with  a  variance  S. 

The  coordinates  are  used  to  generate  the  K  +  1  possible  message  waveforms. 

As  before,  one  of  these  vectors  (called  Xq)  is  selected  and  transmitted,  noise  being 
added  in  the  channel  through  which  transmission  occurs.  At  the  receiver,  the  received  signal 
Xq  +  Y  is  cross-correlated  with  each  of  the  K  +  1  vectors  X^.,  and  the  outputs  are  used  in  ar¬ 
riving  at  a  decision  about  which  of  the  messages  was  transmitted.  An  error  will  occur  in  all 
cases  in  which  the  noise  component  added  to  Xq  is  less  than  (a  -  1)  ^nS  (in  which  case  jXQ|  + 

Xq  *  Y  <anS),  or  in  which  the  component  along  any  of  the  K  other  vectors  exceeds  a  'fnS  (in  which 


case  X, 


X0  +  Xk 


Y  equals  X,  *  Y>anS),  or  any  combination  of  these  events. 


By  the  method 


of  Appendix  IV,  if  the  noise  vector  Y  arises  from  a  Gaussian  distribution,  we  may  take  the  com¬ 
ponents  along  any  of  the  coordinate  axes,  or  along  any  of  the  mutually  orthogonal  vectors,  as  num¬ 
bers  from  the  same  Gaussian  distribution.  Therefore, 

Ik 


P  (no  error)  = 


p  00 

T®M 

/  P(Y)dY 

/  P  (Y)  dY 

'(a- 1 )  |X| 

J  -  00 

r(  a-l)|x| 

poo 

1  -  /  p  (Y)  dY 

\  -  /  P  (Y)  dY 

L  Jalxl  J 

K 


(5-27) 


(5-  27a) 


The  probability  is  the  product  of  the  probabilities  of  individual  events  because  the  coordinates  of 
the  noise  vector  are  independent. 

Since 


Jroo 

p(Y)dY 

a|K| 


should  be  quite  small  in  practical  cases,  it  follows  that 


P  (error) 


oo  i  r  poo 

p  (Y)  dY  1  -  K  / 
(1-a)  |x|  J  L  Ja|x| 


P(Y)  dY 


(5-28) 


Again  neglecting  terms  of  the  second  order  of  smallness,  Eq.  (5-28)  becomes 


♦This  system  and  discussion  is  substantially  the  same  as  that  appearing  in  a  PROJECT  LINCOLN 
internal  memorandum  by  the  author, "Cross  Correlation  Thresholds  and  Channel  Capacity,"  10 
September  1951. 
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y^»  QC  /»30 

P (error)  =  /  p(Y)dY  +  K/  p  (Y)  dY 

J(l-a)|x|  ^  a  |x| 


(5-29) 


When  the  optimum  value  for  a  is  substituted  (and  large  K  and  n  are  assumed),  the  form  of 
Eq.  ( 5 -  Z9 )  is 

n00  i  /  2\  o 00  i  /  2\ 


P  (error) 


if  _L_exp  f-V)  _!_e,p(-4) 

•4(>- s-)v^  1 2'  4k)^^  1  ; 


dv  ,  (5-30) 


The  approximation 


X  exp  (-jt2)dt  =  ^-exP  (-4-)  . 


valid  for  large  Q,  can  be  used  here  when  np  is  large.  Then, 
P  (error)  =  [? - vr.  1  exp  [--l  fl  -  II)  npl 
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-  a 

exp 
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l-cJ 
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’  1  ( 

1  + 

2  - 

(l  +  §)  frTp 

exp 

8  \ 

|  np 

(5-31) 


But  K  has  already  been  assumed  large,  and  if  S«N  so  that  1/2  np  =  C'T  ,  the  expression  becomes 


P  (error)  = 


P  (error) 


.  exp  -lc’T(l  -%Y_  exp  -ic'T(l  +  -g-)2  +  H'T^ 


(l  -  §) 


ttC'T  1  + 


2  f  C'T  ( ,  H\  21 

— -p— ™  exp  -  (l  -  c)  J 


(5-32) 


(5-33) 


The  expressions  given  by  (5-31)  and  (5-33)  are  valid  only  when  the  threshold  coeffi¬ 
cient  is  optimized  (and  subject  to  the  conditions  listed).  A  comparison  of  these  with  (4-30)  and 
(4-31)  reveals  two  differences  of  primary  consideration.  One  of  these  differences,  the  fact  that 
the  ratio  S/P  appears  in  Chapter  IV  while  S/N  appears  in  the  results  just  obtained,  is  easily 
explained  by  noting  that  P  =  N  +  3  includes  the  variation  of  the  magnitudes  of  the  signal  vectors. 
These  vectors  were  of  fixed  length  in  the  example  just  given.  The  other  difference  is  the  pres¬ 
ence  of  the  squared  factor  (1  -  H/C)  in  the  case  of  threshold  detection  while  only  the  first  power 
of  that  factor  appears  in  the  case  of  maximum  correlation  detection.  This  means  that,  for  the 
same  conditions  of  noise,  capacity  and  information  rate,  considerably  greater  delay  is  required 
in  the  case  of  threshold  detection  to  obtain  a  given  probability  of  error  than  when  the  maximum 
correlation  criterion  is  used. 
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CHAPTER  VI 

NOISE  IN  THE  AUXILIARY  CHANNELS 

Up  to  this  point,  the  systems  considered  have  been  those  in  which  some  auxiliary 
noiseless  channels  have  been  utilized  to  make  the  set  of  possible  message  waveforms  available 
at  the  receiver  for  cross-correlation  with  the  received  signal.  Such  systems  are,  of  course, 
special  cases  of  a  more  general  class  of  systems  in  which  the  auxiliary  channels  are  also  dis¬ 
turbed  by  unavoidable  additive  noise* 

The  prime  effect  of  such  disturbing  noise  in  the  auxiliary  channels  is  to  introduce 
some  doubt  as  to  precisely  what  it  is  that  is  being  sought  in  the  cross -correlation  process  at  the 
receiver.  An  additional  uncertainty  remains  after  a  message  has  been  received  and  the  correla¬ 
tion  outputs  obtained  over  that  present  when  the  auxiliary  channels  are  noiseless-  Because  of 
this  greater  uncertainty,  it  is  expected  that  the  decisions  made  on  the  basis  of  these  correlator 
outputs  will  oe  more  frequently  in  error  than  when  the  noise  is  absent.  This  is  found  to  be  the 
case. 

As  in  the  earlier  systems  discussed,  it  is  here  considered  that  the  message  Xn  has 
been  transmitted  through  the  intelligence  channel  in  which  noise  Y  is  added.  Also,  the  set  of 
possible  messages  X^  is  transmitted  in  auxiliary  channels  in  which  noises  Y^  are  added.  It  is 
assumed  that  the  noises  Y^  (k  ^  0)  are  independent  of  noise  Yq  . 

At  the  receiver,  Z  ,  the  received  signal,  is  cross-correlated  with  each  of  the  set  of 

■  ► 

Zk  waveforms  representing  the  corrupted  versions  of  the  possible  message  waveforms.  The 
outputs  resulting  from  the  correlations  are  made  the  basis  of  a  decision  about  which  of  the  mes¬ 
sage  waveforms  was  transmitted. 

At  the  receiver,  the  correlation  of  Z  with  Zk  will  yield 

Z  •  Zk  =  (XQ  +  Y)  ■  (Xk  +  Yr)  ,  (6-1) 

=  XQ.  Xk  +  Y-  (Xk  +  Yk)  +  X0-  Yk  .  (6-la) 

If  the  signal-to-noise  ratio  is  p  in  the  intelligence  channel  and  S/N^  in  the  auxiliary 
channels,  there  will  again  be  an  expected  value  nS  in  the  correct  correlator  output,  while  zero 
is  expected  in  all  the  other  outputs.  Geometrically,  as  illustrated  in  Fig.  6.1,  the  components 

of  Z^  that  fall  along  Z  will  follow  a  Gaussian 
distribution  with  variance  S  +  ,  except 

when  k  -  0  .  For  simplicity,  all  are  con¬ 
sidered  equal  in  the  discussion  following. 

Let  pc  (Z  )  be  the  distribution  of  the 

z  - 

components  of  Zq  along  Z  .  Note  that  Z^  cor¬ 
responds  to  Xz  of  the  systems  discussed  in 
Chapters  IV  and  V. 

When  the  criterion  of  detection  is  that 
of  maximum  correlation,  the  probability  of  error 


Fig.  6.1.  Vector  model  illustrat¬ 
ing  noise  in  the  auxiliary  channel. 
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takes  the  forrr. 


P  (error)  =  1 


(Z  )dZ 
z  z 


p(ZMdZ' 


(6-2) 


where  Z!  is  the  component  of  along  Z.  Of  course,  Z^  is  the  component  of  ZQ  along 

Maximum  correlation  is  used  as  an  example  for  comparison  with  earlier  results. 

It  is  evident  that  the  contribution  to  the  outputs  from  the  correlators  that  do  not  correspond  to 
the  transmitted  waveform  will  be  a  function  of  the  signal  plus  noise  power  rather  than  the 
signal  power  S  alone  This  is  true  independently  of  whether  the  criterion  of  detection  is  that  of 
maximum  correlation  or  exceeding  a  threshold.  Consequently,  the  final  result  obtained  for 
noise  added  in  the  auxiliary  channel  can  be  extended  readily  to  threshold  detection  by  analogy. 

The  distribution  (Zz)  is  peaked  about  its  average  value  in  much  the  same  manner 
as  p^(Xz)  .  This  average  value  may  be  obtained  as  follows.  The  expected  output  is  NS  .  The 
average  magnitude  of  is  vnP  .  Thus,  although  Zq  and  Z^  are  not  independent,  the  dis¬ 
tributions  of  these  quantities  for  large  n  are  sufficiently  peaked  that,  to  a  good  approximation. 


ave  Z 

z 


(6-3) 


The  behavior  of  the  probability  of  error,  again  under  the  assumption  of  large  n  , 
may  be  obtained  by  evaluating  Eq.  (6-2)  at  the  average  value  of  Z^  .  Therefore, 


P  (error)  ^  K 


P 


2  1 

exp 

- 1 

CM 

_L_i 

K 


V  2.Pfe 

nS2 

/— 

JnS 

2Ppk 

(6-4) 


(6  -4a) 


Again  use  is  made  of  the  approximations  for  p  «  1  and  K  »1  which  are,  respec¬ 
tively,  l/2np  =  C'T  and  InK  =  H’T  .  Equation  (6-4a)  becomes 


P  (error) 
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5 


2%/  ttC'T  v  s 

_ I _ 

zj  IT  JE  C'T 


exp 


—  C  rT 


H 

pk~c 


exp 


-*) 


(6-5) 


(6-5a) 


Note  that  for  -  0  ,  S/P^  =  1  ,  and  Eq.  (6-5)  reduces  to  the  result  obtained  in 
Chapter  IV  see  Eq.  (4-31)  .  Furthermore,  to  obtain  arbitrarily  small  probability  of  error. 
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with  increasing  T  ,  it  is  indicated  that  the  ratio  H/C  should  be  less  than  the  signal-to-noise 

ratio  in  the  auxiliary  channels.  The  appearance  of  {S/P^)C'  in  each  place  where  C1  appears 

in  Eq.  (4-31)  indicates  that  the  effective  system  capacity  is  S/P^  times  the  theoretical  capacity. 

It  may  be  expressed  by  stating  that  communication  is  possible,  relatively  speaking,  to  the  extent 

* 

that  noise  is  absent  in  the  auxiliary  channels. 


*It  is  suggested  that  transmission  of  the  "code  book,  "  which  is  somewhat  akin  to  learning,  must 
be  accomplished  perfectly  if  the  information  presented  in  that  code  is  to  be  transmitted  at  a  rate 
equal  to  the  capacity  of  the  channel.  Also,  since  it  is  evident  that  some  form  of  auxiliary  chan** 
nel  must  be  used  for  what  amounts  to  code-book  transmission,  the  total  equivalent  capacity  can 
never  be  utilized  perfectly. 
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CHAPTER  VII 


THE  EXPERIMENTAL  STUDY  OF  PROBABILITY  OF  ERROR 


A.  The  Modified  Theory  for  K  -  i 


In  the  experimental  verification  of  the  theoretical  material  presented  in  this  paper, 
a  relatively  simple  physical  system  was  constructed.  In  this  system,  one  possible  message 
waveform  is  generated  in  the  transmitter  and  sent  through  an  auxiliary  channel  to  the  receiver. 
The  waveform  is  also  sent  through  the  intelligence  channel,  modulated  in  an  on>off  manner,  and, 
in  the  process  of  transmission,  disturbing  noise  is  added.  A  simplified  block  diagram  of  the 
system  is  shown  in  Fig.  7.1,  and  a  detailed  description  of  the  actual  system  follows  in  a  later 
section  of  this  chapter.  The  system  allows  for  the  possible  introduction  of  a  disturbing  noise  in 
the  auxiliary  channel  also. 


(Auxiliary  Channel) 


Fig.  7.1.  Simplified  block  diagram 
of  the  binary- choice  system. 


It  is  understood  that  in  this  system  K,  +  1  =  2,  orK-I,  the  alternative  signal  to  the 
waveform  generated  at  the  transmitter  being  a  zero  or  null  waveform  corresponding  to  the  key- 
up  position.  In  the  experimental  system,  the  on-off  indicator  at  the  receiver  operates  to  indi¬ 
cate  "on”  whenever  a  preset  threshold  is  exceeded  by  the  correlation  receiver* 

Certain  simple  modifications  of  the  analysis  underlying  the  general  case  for  large 
K  are  required  to  theoretically  describe  this  NOMAC  system.  Here,  an  error  is  made  when 
either  the  correlation  for  the  key- up  position  exceeds  the  threshold  or  when,  for  the  key-down 
position,  the  threshold  is  not  exceeded. 

Let  the  threshold  be  set  at  X*.  On  the  average,  the  probability  of  error  is  given  by 

P  (error)  =  P (on)  [P  ("X'2  +  |X|  <  X*)]  +  p(off)  [P(|X|  Y,  >  X*)]  .  (7-1) 

This  equation  can  be  expressed  as  the  average  over  all  'X|  of  the  probability  of  error  when  |X; 
is  fixed.  Thus, 


P  (error) 
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L  2N  !x| 


V211N  |X; 
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(7-2) 


If  the  system  is  being  used  efficiently,  the  symbols  "‘on11  and  "'off’  will  occur  with  equal  proba¬ 
bility.  Then, 
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P  (error)  =  \  J  p  (|X>)  d  |X||  f  ,  *  — j= 

dJo  °  KX|-X  V2» 


2  1  2  1  1 

- — *,  exp - - -p 

znjxeL  *  r 00 _ t w  L. 


X*  'JUrN  |x|  j 


(7-2a) 


A  choice  of  X*.  the  threshold,  is  made  to  minimize  the  probability  of  error.  It  follows  that  the 
solution  of 

— ^  P(error)  =  0  (7-3) 


is  taken  for  the  threshold.  A  solution  of  Eq.(7-3)  is  a  solution  of 


V  2--N  |X| 


1  c~d\  <i*2-x*)2 

= eXP  - T — 


1  f  (X*)2 

= exp 1 - - — 

rN  :x!  L  2N  ;Xj  ‘ 


|x|2  -  X*  =  X*  ,  X*  =  . 


where  ,X  is  fixed. 

The  threshold  value  as  considered  here  is  a  constant  and  not  a  function  of  |x|  .  To 
obtain  that  constant,  X*  is  averaged  over  all  ,X|  and  the  result  nS/2  is  obtained-  This  is  the 
constant  value  of  optimum  threshold  used.  Then, 
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{k  jX|i 


(7- 6a) 


The  probability  distribution  function  p^flX)  is  the  cni-squared  type  discussed  in  de¬ 
tail  in  Appendix  II. 

If  the  substitution  |X.  ~  a  *{nS  is  made,  Eq.(7-6)  reduces  to  a  more  convenient  form 
(in  which  Stirling's  approximation  for  P(n)  has  been  used),  namely. 


P  (error)  =  '/njT  a"  1  ex*4“  ~  1)]{lerf  /1/^}  da 


Erf(Q)  is  defined  here  as 


rrr  r 

*V= JQ  exp[-^-Jdt 


Another  form  of  Eq.(7-7),  using  the  nomenclature  of  Fano,  reads 


'(error)  =  a"’1  exp[- -|(a2-  erf 


da 


(7-7a) 
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The  argument  of  the  error  function  is  then  the  square  root  of  the  modified  ratio  of  the  signal  en- 
ergy  to  the  noise  power  per  cycle* 

Curves  of  the  probability  of  error  for  several  typical  values  of  time -bandwidth  prod¬ 
uct  and  signal- to-noise  ratio  are  given  in  Fig.  7.2.  These  results  were  obtained  by  numerical 
integration  of  Eq.(7-7),  and  the  method  as  well  as  tabulated  results  are  shown  in  Appendix  VL 
The  probability  of  error  is  seen  to  be  primarily  a  function  of  the  product  np  ,  and  is  plotted  as 
a  function  of  that  parameter  in  Fig.  7.3.  Over  the  range  of  values  of  np  for  which  the  plot  is 
given  in  Fig.  7.3,  the  deviation  from  the  single  line  is  less  than  the  line  width  of  the  curve  for 
n  =  1000  when  n  -  100  and  more.  Where  its  values  differ  from  the  other  curve  by  an  appreci¬ 
able  amount,  the  separate  curve  for  n  =  10  is  shown. 


Fig.  7.2.  Probability  of  error  as  a 
function  of  signal -to -noise  ratio  (p)  and 
system  bandwidth  ratio  (n). 


Fig.  7.3.  Probability  of  error  as  a 
function  of  np  . 


When  it  is  recalled  that  np  represents  the  signal-to-noise  ratio  at  the  output  of  a 
correlation  detector  (as  well  as  the  ratio  Es/Nq),  the  significance  of  the  fact  that  the  probability 
of  error  is  primarily  a  function  of  this  output  ratio  becomes  apparent.  It  justifies  the  assump¬ 
tion  that  the  noise  component  of  the  output  signal  is  Gaussian,  because  the  curve  as  a  function 
of  np  is  essentially  that  approached  by  Eq.(7-7a)  for  n  large,  which  is  in  turn  the  error  function 
evaluated  at  a  =  1  *  The  error  function  is,  of  course,  expressed  in  terms  of  the  Gaussian  or 
normal  distribution. 


B.  Description  of  the  Equipment 

In  its  physical  form,  the  equipment  required  to  investigate  the  validity  of  the  theoret¬ 
ical  results  consists  of  five  panels  mounted,  together  with  power  supplies,  on  a  single  relay 
rack.  In  addition,  the  Davenport  probability-measuring  equipment^  was  used  to  make  actual 
measurements  of  the  results  experimentally.  The  complete  experimental  setup  is  pictured  in 
Fig.  7.4. 

On  the  relay  rack  containing  the  NOMAC  system  (at  left  in  the  picture),  the  following 
are  mounted  (from  top  to  bottom):  low- voltage  pow'cr  supply,  random  (noise -like)  signal  source, 
side-band  generator  (transmitter),  the  simulated  channel,  the  correlation  converter,  the 
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integrator-detector  (receiver),  the  channel-noise  power  supply,  and  the  receiver-transmitter 
power  supply.  The  interconnection  of  components  other  than  power  supplies  is  shown  in  the 
block  diagram  (Fig.  7.5). 


The  schematic  diagram  of  the  random -signal 
source  is  given  in  Fig.  A7.2  of  Appendix  VII.  Essentially,  it 
is  a  filter  amplifier  with  a  bandwidth  of  5  megacycles  and  a 
midband  frequency  of  35  megacycles.  The  amplifier,  featur¬ 
ing  an  8-pole  Tchebychef  frequency  response,  was  designed 
and  built  by  Ft.  Price  and  W.  McLaughlin  of  Group  34  of 
PROJECT  LINCOLN.  The  gain  of  the  8  stages  (6BH6's)  is 
sufficient  to  amplify  the  thermal  noise  and  tube  noise  of  the 
first  stage  to  an  output  power  of  about  3  milliwatts  of  white 
noise  into  a  75 -ohm  load  impedance.  The  noise  output  3  Me 
from  the  midband  frequency  is  down  50  decibels  or  more 
from  the  level  within  the  pass  band. 

The  side-band  generator  or  transmitter  is  shown 
schematically  in  Fig.  A7.3.  The  signal  input  from  the  random 
signal  source  is  mixed  with  a  5.35 -Me  signal  from  the  crystal 
controlled  local  oscillator.  Separate  2-stage  band-pass  am¬ 
plifiers  select  the  resulting  side  bands  at  29.65  Me  and  40.35 
Me.  Each  of  these  stagger-tuned  amplifiers  is  adjusted  to 
an  effective  bandwidth  of  2.0  Me,  and  fed  through  cathode  followers  to  the  simulated  channel. 

The  resulting  side-band  spectra  have  rounded  corners  so  that  the  transition  from  channel  noise 
to  channel-plus-signal  noise  will  be  a  smooth  function  of  frequency,  and  thus  less  apparent  to  an 
unfriendly  receiver. 

The  simulated  channel  consists  of  attenuators  for  the  two  signals  and  for  the  noise 
added  in  as  the  corrupting  or  disturbing  noise.  Also,  an  adding  circuit  is  used  to  obtain  the  sum 
of  the  proper  amounts  of  random  signal  and  contamination  noise.  In  th£  first  part  of  the  experi¬ 
mental  work,  the  40. 35 -Me  signal  is  connected  directly  to  the  correlator  (as  the  reference  sig¬ 
nal)  to  simulate  a  storage  process  which  makes  a  noise-free  version  of  the  signal  available  for 
cross -correlation.  In  the  second  part,  noise  is  added  in  the  40.35-Mc  channel  also,  to  simulate 
the  conditions  when  both  reference  signal  and  intelligence  signal  must  be  transmitted  through  a 
noisy  medium.  The  schematic  diagram  is  shown  in  Fig.  A7.4.  The  two  sources  of  disturbing 
noise  are  modified  radar  IF  strips,  originally  built  for  the  M.I.T.  Radiation  Laboratory.  When 
operated  at  full  gain  and  with  no  input,  these  strips  have  a  noise  output  of  approximately  one 
volt. 


Fig.  7.5.  Block  diagram  of 
the  experimental  system. 


The  correlation  converter  comprises  a  pair  of  wide -band  RF  amplification  stages, 
one  each  at  29.65  Me  and  40.35  Me*  and  a  6AS6  used  as  a  multiplier.  The  choice  of  the  6AS6 
was  made  because  the  same  bias  on  grids  one  and  three,  namely,  -2.5  volts,  places  the  tube  in 
an  operating  range  leading  to  a  minimum  of  distortion  in  the  multiplication  process.  Thus  the 
biasing  problem  was  simplified.  The  output  of  this  tube  is  fed  to  a  two-stage  10.7  Me  IF  ampli¬ 
fier  using  conventional  components. 
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The  frequency  of  the  IF  amplifier  is  chosen  as  follows:  when  the  signals  at  the  grids 
of  the  6AS6  are  n^tjcos  [<^t  +  9^  ( t) ]  and  n^(t)cos[w^t  +  e^t)]  *  the  Pr°duct  is  given  by 

+12(t)  =  nj(t)n2{t)  j  jcos[(«r  «j)t  +  02(t)  -  O^t)]  +  cos  [(u>2  +  t  +  02(t)  + 

(7-8) 

Now  if  and  as  well  as  the  phases  0^  and  0^  are  independent,  (n^(t)  and  n^(t)  have  zero 
average),  the  average  of  n^n^  *s  zero  an<^  phase  of  the  term  at  frequency  is  random. 

However,  if  and  0^  =  0^  ,  there  is  no  random  phase  in  the  difference-frequency  term  and 

the  product  n^(t)^  has  an  average  proportional  to  the  power  (or  variance).  Thus,  in  the  experi¬ 
mental  model,  an  output  of  the  multiplier  at  the  difference  frequency  of  10.7  Me  should  occur 
whenever  the  two  side  bands  generated  in  the  transmitting  process  are  applied  to  the  two  signal 
grids.  Figure  A7.5  is  the  schematic  diagram  of  the  correlation  converter. 

The  schematic  diagram  of  Fig.  A7.6  shows  that  the  integrator-detector  is  merely  a 
double -conversion  superheterodyne  receiver  using  a  crystal-controlled  oscillator  at  the  first 
converter  and  a  Collins  Type  70E-15  oscillator  for  the  second- converter  oscillator.  The  IF  fre¬ 
quency  is  455  kc  and,  with  a  crystal  filter  at  the  same  frequency,  integration  bandwidths  of  the 
order  of  80  cycles  are  obtained.  In  the  broad-band  (crystal  out)  position,  the  bandwidth  is  4000 
cycles.  The  former  bandwidth  is  typical  of  the  requirements  for  teletype  circuits,  wnile  the 
latter  is  adequate  for  speech  communication.  Filters,  as  integration  devices,  are  discussed  in 
Chapter  VIII. 

C.  Discussion  of  Experimental  Results 

An  examination  of  Eqs.(7-2a)  and  (7-6)  reveals  that,  for  the  conditions  imposed,  the 
probability  of  error  for  "on-off”  signaling  is  the  same  as  the  relative  frequency  of  errors  in  re¬ 
ceiving  only  "on"  signals,  since  the  individual  integrals  contribute  equally  to  the  probability  of 
error.  This  somewhat  simplifies  the  experimental  procedure  in  that  measurements  or  counts 
made  with  the  signal  present  in  the  intelligence  channel  can  be  taken  as  the  desired  probability 
of  error. 

The  probability- distribution  amalyzer  developed  by  W.  B.  Davenport,  Jr,  was  used  to 
count  the  relative  number  of  times  the  output  signal  exceeded  the  threshold  value  which  was  set 
at  one -half  the  average  output  value.  The  difference  between  this  relative  count  and  unity  was 
taken  as  the  experimental  probability  of  error. 

Curves  were  taken  with  the  signal -to -noise  ratio  at  the  input  as  the  independent  vari¬ 
able,  and  various  values  of  the  signal -to -noise  ratio  in  the  auxiliary  channel  as  a  parameter  for 
the  two  values  of  bandwidth  ratio.  When  the  auxiliary- channel  signal- to -noise  ratio  is  infinite, 
the  measured  values  of  probability  of  error  should  check  the  theoretical  probability  of  error  as 
obtained  by  numerical  integration  and  presented  in  Fig.  7.2.  When  noise  is  added  in  the  auxiliary 
channel,  the  effect  as  predicted  in  Chapter  VI  should  be  observed.  This  effect  is  to  change  the 
effective  value  of  np  in  the  intelligence  channel  by  the  factor  S/(N^+  S),  This  means  the  theoret¬ 
ical  probability-of -error  curves  should  be  shifted  by  an  amount  corresponding  to  that  factor. 

The  curves  in  Figs.  7.6,  7.7,  and  7.8  show  the  results  of  the  experimental  measure¬ 
ments.  Figure  7.6  shows  the  probability-of -error  measurements  made  with  the  wide -band  filter. 
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The  value  of  n  calculated  for  this  filter  is  615,  based  on  the  equivalent  noise  bandwidths  of  the 
input  (RF)  and  output  [IF)  filters.  The  measurements  were  made  for  probability  of  error  less 
than  10  per  cent  which  corresponds  to  an  output  (postcorrelation)  signal-to-noise  ratio  of  about 
+8db.  This  upper  limit  was  established  more  or  less  arbitrarily  because  of  the  type  detection 
used,  i.e.,  envelope  detection,  after  multiplication  and  filtering.  The  level  of  8  db  represents  a 
signal-peak- voltage  to  rms -noise -voltage  ratio  of  about  2.5,  for  which  the  number  of  excursions 
of  noise  voltage  which  exceed  the  peak  signal  voltage  is  of  the  order  of  one  per  cent  if  the  noise 
is  approximately  Gaussian,  and  the  portion  of  time  spent  by  the  combined  signal  and  noise  volt- 
age  in  the  curved  portion  of  the  detector  characteristic  is  correspondingly  small.  Figure  7.7 
shows  similar  results  for  the  narrow-band  filter  which  has  a  measured  n  of  31,700.  In  prac¬ 
tice.  this  filter  is  so  narrow  that  the  drift  of  the  crystal-controlled  oscillators  is  a  serious  prob 
lem,  ai^ri  readings  are  difficult  since  all  tuning  and  level  settings  must  be  corrected  before  a 
count  and  then  checked  for  drift  after  the  count.  For  this  reason,  only  one  additional  curve  for 
noise  in  the  auxiliary  channel  was  taken  to  show  again  the  predicted  displacement.  It  is  to  be 
assumed  that  the  effect  shown  in  Fig.  7.6  would  have  appeared  here  also  for  other  values  of  auxil 
iary -channel  signal-to-noise  ratio. 

Figure  7.8  shows  the  results  of  the  earlier  figure,  plus  some  additional  counts,  as 
a  function  of  "effective  np,"  where  the  effective  value  of  np  is  defined  as  n(S/P)aux.  The  com¬ 
posite  theoretical  curve  is  given  for  comparison  of  theoretical  with  experimental  results. 

The  spread  of  the  points  in  Fig.  7.8  is  attributed  to  two  principal  sources  of  error. 
The  ability  to  match  signal  and  noise  powers  was  limited  by  the  resolution  of  the  attenuators, 
which  had  a  range  from  zero  to  101  decibels  attenuation  in  steps  of  one  decibel.  The  accuracy 
of  components  used  in  the  attenuators  was  5  per  cent,  but  in  general  the  accuracy  of  any  given 
step  was  considerably  better  than 'that.  However,  the  absolute  error  introduced  by  the  larger 
steps  was  observed  to  be  as  much  as  0.4  decibels. 


Fig.  7.6.  Experimental  probability  of  error  vs.  -signal-to-noise 
ratio,  wide-band  integrating  filter. 
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RATIO  np  (S/P)aux  (decibels) 


Fig.  7.7.  Experimental  probability  of 
error  vs.  signal-to-noise  ratio,  narrow-band 
integrating  filter. 


Fig.  7.8.  Theoretical  and  experimental 
results  of  probability  of  error  as  a  function 
of  effective  np. 


A  second  source  of  error  was  that  of  reading  the  meters  used  in  setting  output  levels. 
This  process  was  made  more  difficult,  since  the  output  signals  contained  noise  with  significant 
low-frequency  components. 

The  average  of  the  observed  results  taken  experimentally  shows  a  shift  to  the  left  of 
approximately  one-half  a  decibel.  Perhaps  the  principal  contribution  to  this  shift  (amounting  to 
some  12  per  cent)  was  the  error  in  measuring  the  bandwidth  of  the  input  signals  at  the  correlation 
receiver.  In  order  to  avoid  such  possible  sources  of  error  as  saturation  in  amplifiers  and  Miller 
effect,  the  measurements  were  made  at  low  level  from  the  frequency- response  pattern  appearing 
on  an  oscilloscope.  Under  this  condition,  the  noise  output  of  the  amplifier  was  comparable  to  the 
amplitude  of  the  response  pattern.  Consequently,  the  amount  of  error  apparently  present  does 
not  appear  to  be  inconsistent. 

Still  another  source  of  error  undoubtedly  contributed  to  the  spread  of  experimental 
results  and  to  the  relative  increase  of  the  probability  of  error  for  low  values  of  probability.  The 
presence  of  high-intensity  radiation  from  various  sources  (e.g.,  radar)  in  and  around  the  labora¬ 
tory  was  observed  to  cause  distinct  errors  in  the  counts  made  by  the  probability  analyzer.  An 
attempt  to  minimize  these  errors  was  made  by  taking  counts  at  times  when  most  other  laboratory 
functions  were  closed  down.  The  errors  due  to  outside  sources  were  kept  small  enough  to  be 
consistent  with  other  experimental  errors  in  the  range  over  which  measurements  were  taken, 
but  they  were  increasingly  significant  at  very  low  probability  of  error,  and,  in  fact,  set  a  prac¬ 
tical  lower  limit  to  the  range  of  observation. 

It  should  be  mentioned  that  each  point  plotted  represents  an  average  of  several 
counts,  and,  where  two  or  more  points  are  plotted  in  Figs.  7.6  and  7.7  for  a  given  condition,  they 
represent  readings  taken  on  different  days  and  under  possibly  different  conditions  of  laboratory 
temperature,  noise  level,  and  equipment  adjustment.  They  are  plotted  separately  to  indicate 
the  reproducibility  of  the  results. 
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CHAPTER  VIII 


RELATED  TOPICS  TO  NOMAC  SYSTEM  DESIGN 


A,  The  Effect  of  Nonideal  Integration 

The  validity  of  the  theoretical  results  thus  far  obtained  is  limited  by  the  accuracy  of 
the  initial  assumptions,  as,  for  example,  true  Gaussian  amplitude  distributions,  ideal  short- 
time  integration,  distortionless  multipliers,  and  rectangular  pass  bands.  However,  the  effects 
of  certain  of  these  factor's  being  less  perfect  than  assumed  can  be  ascertained,  and  to  some  ex¬ 
tent  quantitative  corrections  can  be  made.  Since  experimenters  have  found  that  most  natural 

noise  sources  (thermal  agitation,  shot  noise,  etc.)  do  have  a  Gaussian  amplitude  distribution  at 

14 

least  for  amplitudes  less  than  about  six  times  the  standard  deviation,  it  seems  evident  that,  if 
physical  systems  are  operated  in  such  a  manner  that  saturation  and/or  cut-off  effects  may  be 
neglected,  the  assumption  of  Gaussian  noise  seems  well  founded.  Considerably  larger  differ¬ 
ences  may  occur,  however,  if  the  integration  method  (filtering)  is  nonideal  or  if  the  multiplier 
used  in  the  correlation  process  introduces  appreciable  distortion. 

To  investigate  the  effect  of  nonideal  integration,  it  will  be  assumed  that  a  correlation 
output  W  results  from  the  sum  of  weighted  components  w^ .  Here  is  a  product  com¬ 

ponent  of  the  type  obtained  when  the  functions  X(t)  and  Z{t)  are  represented  in  the  form  indi¬ 
cated  by  Eq.  (1-6).  The  results  are  limited  to  low-pass  functions  as  derived,  although  the  final 
result  agrees  with  results  obtained  for  band-pass  functions  when  viewed  from  a  different  ap¬ 
proach.^ 

The  weight  associated  with  each  is  h.  ,  and  the  h's  are  constants  and  independent 
of  the  w's  .  Thus 

n 

W  =  E  h.w.  .  (8-1) 

ii 


Obviously,  if  all  hr  equal  unity,  the  W  resulting  is  that  discussed  earlier  under  the 
assumption  of  ideal  integration. 

Now, 

—  f  i  (  ) 

(8-2) 


W  =  E  |  Zh / w^j  =  2  E 


=  W  Shi 
i 


Similarly, 


=  E  )  Eh.  w.  2h.wl 

1  i  1  1  j  J  * 

-  *  *  hih,  e  M 


But  is  independent  of  for  i^4j,  so  that  one  obtains 


W 


■ *  *  hihi  R ‘1  6}'. ft 


(8-2a) 

(8-3) 

(8-3a) 

(8-4) 


where  6^  is  the  Kronecker  delta. 
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From  the  knowledge  above  of  W  and  W2  ,  the  variance  tr  may  be  obtained.  Thus 


?2  2 
<r  =  W  -  W  =  2 
w 


2  2  h.h.  i  w ?  6^  +  ( 1  —  6*!  )  w .  ^  -  W  2  £  £  h.h. 
i  j  1  J  l  1  1  1  1  /  1  i  j  1  J 


-  [w  ?  -  w  .^1  2h.^  =  o*  ^  2  h2 

L  l  l  J  i  l  w  1 


(8-5) 

(8-6) 


Again,  Eq.(8-6)  is  seen  to  reduce  to  the  case  presented  earlier  when  all  hr  equal  unity,  i.e.f 
ideal  integration. 

Since  the  errors  in  the  systems  considered  here  are  caused  by  fluctuations  about  the 
average  value  of  the  output,  the  ratio  of  importance  is 


JW 


J. 


2h. 


w 


J—  Eh2 
V  n  l 


i  * 

1  w 


W 


w 


2h. 

i 


^/tl  w  —  2h. 

n  i 


\Tn  w 


(8-7) 


(because  an  rms  value  is  never  less  than  the  average  value)*  But  the  term  on  the  right  is  the 
result  obtained  with  ideal  integration.  Thus,  there  will  be  more  frequent  errors  if  nonideal  in¬ 
tegration  is  used. 

To  apply  this  result,  the  probability  of  error  expression  Eq.  (4-30)  is  examined. 

Here 

P  (error)  ~ 


K  fZ  eXP  r-— 1 

nS  p  L  2PJ 


\f~2ir' 

In  this  expression  S  is  cr  *  and  the  average  W  is  given  by  S  \f n/p.  If,  instead,  the  values  for 
2  _  w 

cr  yy  and  W  that  would  result  from  nonideal  integration  are  substituted,  the  expression  becomes 


P (error)  ~ 


K 
V  2tt 
2/^2 


IE 

7  2  h.2 

S 

(Sh.)2' 

ip 

ca}J 

2h. 

l 

*  2P 

Eh.2 

1  J 

(8-8) 


The  ratio  (2h.)  /21n  appears  twice  and  by  definition  will  be  the  effective  n*  Since  n  is  twice 
the  time -bandwidth  product,  the  effective  n  is  thus  defined  in  terms  of  2rW  (this  W  is  band¬ 
width),  where  t  is  the  effective  integration  time.  It  follows  by  definition  that 


,  if.y 


2W 


*hi 

l 


(8-9) 


If  the  approximations  nS/2N  =  C'T  and  InK  =  H’T  ,  valid  for  K  »  1  ,  n  »  1  ,  and 
S  «  N  are  used  here,  as  in  Chapter  IV,  the  expression  for  probability  of  error  becomes 


or 


P(error)  ^  - 


2 \l  TT  C'T 


exp  ~C't  +  H'T 


(8-10) 


P(error)^ 


2  J  IT  (^)  C'T 


exp 


(8-10a) 
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Here  {t/T)C‘  appears  in  each  place  where  C'  appeared  in  Eq.  (4-30).  It  is  interpreted  as  a  de¬ 
crease  of  effective  system  capacity  corresponding  to  the  factor  t/T  which  will  never  exceed 
unity,  since 

(,yr 

n  2 
2  h 

i=l  1 

is  less  than  n.  This  in  turn  is  true  since  the  time  devoted  to  signaling  must  be  at  least  as  long 
as  the  time  devoted  to  averaging  the  correlated  signal. 

The  integration  time  can  be  obtained  from  the  impulse  response  function  of  the  inte¬ 
gration  filter.  One  notes  that  the  weights  In  themselves  are  values  of  the  impulse  response 
function  at  time  intervals  At  =  l/2w.  Therefore 

si>i=a(^)  (8-n) 

i  i 

where  h  (t)  is  the  filter  impulse  response.  Furthermore,  (1/V.w)  is  given  approximately  by 
-T 

2W  /  h(t)dt  . 


T 


Therefore,  the  effective  integration  time  is  given  by 
h  (t)  dt  2  [/  h  (t)  dt  2 


(8-12) 


(2W)2  f 

Jo 


h(t)  dt 


/  - 


w2  dt 


Equation  (8-12)  is  based  on  the  assumption  that  h(t)  is  zero  for  all  t  greater  than 
T  .  This  implies  that  t  <  T  *  Again,  it  is  noted  that,  for  an  ideal  integration  function  h(t)  =  1 
for  0  <  t  <  T,  and  zero  elsewhere, t=  T  ,  and  the  result  is  the  same  as  that  obtained  in  Chapter  IV. 

Equation  (8-12)  may  be  manipulated  in  the  following  way:  the  difference  between  the 
integral  with  upper  limit  T  and  upper  limit  oo  should  be  zero  (or  at  least  negligible). 

Then 

1 

i(t)  dt 

TT 

(8-13) 


.  IT" 


H  (o) 


r hw2dt  -f  r  iH(*)i2dc 

J0  £nJ.  oo 


H  (oj 


/i oo 

■on 


(8-13a) 


H  (f )  I  df 


This  is  obviously  the  reciprocal  of  what  is  defined  as  the  effective  or  noise  bandwidth.  Thus 

.  1 


■aT 


(8-14) 
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The  noise  bandwidth  Af  is  defined  for  double-sided  spectra  (defined  for  -co  <  f  <oo).  Thus,  the 
input  noise  bandwidth,  since  rectangular  channel  pass  bands  are  assumed,  is  2W  .  This  means 

2W 


Af 


(8- 14a) 


the  ratio  of  channel  noise  bandwidth  to  integrating  filter  noise  bandwidth.  This  result  is  obtained 
from  a  different  approach  in  Ref.  4. 

B.  The  Effect  of  Distortion  in  Multipliers 

The  correlation  process  inherently  implies  multiplication,  yet  electronic  multiplica¬ 
tion  of  two  time -varying  parameters  is  not  easily  accomplished.  Probably  the  simplest  type  of 
electronic  multiplication  is  that  which  takes  place  in  the  conventional  converter  stage  of  a  super¬ 
heterodyne  receiver,  and  is  the  type  used  in  the  correlation  converter  of  the  experimental  sys¬ 
tem  . 

The  first-order  analysis  of  tubes  of  the  type  used  as  multipliers  is  that  the  output 
current  is  the  product  of  two  quantities,  a  +  f ^(t)  and  b  +  (t)  .  The  result  is,  of  course, 

ab  +  bf^(t)  +  af^  (t)  +  fj(t)f^(t).  The  constants  a  and  b  as  well  as  the  components  bfj(t)  and 
af^  (t)  may  be  separated  from  the  product  f  ^ (t)  f ^  (t)  by  band-pass  filtering,  if  fj(t)  and  (t) 
are  functions  occupying  distinct  frequency  bands.  Thus,  in  view  of  the  elementary  analysis,  the 
converter-tube  multipliers  are  as  good  as  the  filtering  used  to  separate  the  desired  component 
from  the  undesired  ones. 

However,  a  more  general  approach  considers  the  effects  of  distortion  in  the  multi¬ 
plying  process.  In  general,  the  time  function  of  the  output  may  be  expressed  by  the  summation 


+  (t)  =  2  S  a..f,1  (t)f J(t) 

i=o  j=o  1J  1  c 


(8-15) 


The  iJj (t)  thus  obtained  is,  for  example,  the  plate  current  of  the  converter  tube  to 
which  f ^ (t)  is  applied  at  one  grid  and  f ^  (t)  is  applied  at  another.  It  is  further  assumed  that  the 
spectra  of  fj(t)  and  (t)  do  not  overlap.  The  term  yielding  the  desired  output  is  that  associated 
with  ,  and  in  particular,  one  of  the  two  side  bands  -  say,  that  at  the  difference  frequency  - 
is  isolated  by  filtering.  Thus,  the  desired  output  is 


+d(t)*"2lfl(t)  f2(t) 


(8-16) 


The  products  of  distortion  that  lie  in  the  product-frequency  band  are  the  only  ones 
that  will  lead  to  errors  in  the  results.  How  these  occur  and  their  relative  significance  can  be 
obtained  by  considering  as  typical  functions  those  defined  by 


Then 


f  1  (t)  =  Xj  COS  Wjt 

;  f^  (t)  =  cos 

u>2l  . 

(8-17) 

[cos  U),t]r  =  -Lexp  [-jru.t] 

r  r 

E  rCkexpl 

2jk<V"l  ’ 

(8-18) 

L  1  J  2/  L  J 
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J 
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1  r  . 

—  exp.^-jsu2t 


£  sCh  exp  J^jho^t 


(8-  18a) 


From  these. 


TtW.2  ,2  rCk  SC  exp  [jt  (2k-r)Wl+(2k-s)u>2 
2  k-o  h=o  u 


(8-1  8b) 


Now,  if  the  filtering  is  such  that  only  the  component  at  frequency  -u^  lies  within  the  pass 
band,  only  those  terms  for  which  2k- r  =  1  and  2h  —  s  =  —1  will  be  passed  by  the  filter.  For 
these  conditions. 


k  = 


r  +  1 
2 


(8-19) 


Therefore,  the  component  at  the  difference  frequency  that  will  appear  at  the  output  of  the  filter 
will  be 


r  (r+l)/2  s  is-l)/2 
->r+s  -  1 


COS  (o 


1 


“2>l 


(8-20) 


It  is  noted  that  (r  +  l)/2  and  (s  -  l)/2  must  be  integers,  and  that  consequently  r 
and  s  must  be  odd  integers.  Furthermore,  because  of  the  symmetry  of  the  binomial  expansion, 
s^(s-l)/2  =  s^(s  +  l)/2  anc*  *ke  *atter  may  substituted  in  the  given  expression  for  greater 
symmetry.  Thus,  where  the  functions  cos  Wjt  and  cos  u^t  differ  by  a  third  frequency  not  har¬ 
monically  related  to  either  of  the  other  two,  the  expression  for  the  portion  of  the  product  func¬ 
tion  appearing  at  the  output  of  the  filter  is 


oo 
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i=l 
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(8-21) 


Here  x^  and  are  the  constants  by  which  sinusoidal  terms  cos  c^t  and  cos  w^t  are  multiplied, 
and  the  expression  (8-21)  shows  the  contribution  of  distortion  terms  in  a  converter -multiplier 
using  filtering  in  the  output. 

As  thus  derived,  and  x^  may  be  thought  of  as  quasi-stationary  coefficients  — 
that  is,  Xj  and  x^  may  vary  with  time  if  those  variations  are  slow  compared  to  the  frequency 
representing  the  filter  bandwidth.  Obviously,  this  leads  to  contradictions  if  the  signals  being 
analyzed  are  limited  in  their  fluctuations  by  the  response  of  the  filter.  However,  the  exact 
analysis  is  sufficiently  more  difficult  to  justify  using  expressions  such  as  (8-21)  as  a  guide  to  the 
effects  of  multiplier  distortion. 

It  is  noted  in  passing  that  the  coefficients  a.^  are  related  to  the  coefficients  of  a  Tay¬ 
lor  expansion  by  the  expression 


»  9I+J  d.  (t) 

af,(.)‘af2(t)J  't.t0  ' 

from  which  one  obtains 

+  [fl(to  +  t} ,  f2  +  l)]  =  aoo  +  aol  f2  (t)  +  alo  fl(t)  +  •  •  • 


(8-22) 


(8-23) 


Experiments  conducted  by  R.S.Berg  and  B.Eisenstadt  (as  yet  unpublished)  have 
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shown  that,  for  a  reasonable  choice  of  tube  operating  point,  most  converter  tubes  tested  had  con¬ 
tributions  due  to  j  and  terms  40db  or  more  below  the  desired  terms.  Higher -order 

terms  were  found  to  be  appreciably  smaller. 

Therefore,  it  is  found  that  (3/8)a^  <  .01  ( 1/2  a^) ,  where  the  coefficients  3/8  and 
1/2  are  the  values  obtained  from  Eq.  (8-20).  The  numbers  a.,  are,  of  course,  those  that  might 
be  obtained  from  an  exhaustive  analysis  of  the  static  characteristics  of  the  converter  tube.  It 
is  further  noted  that  the  effective  values  of  the  terms  of  6th  order  are  (5/32)  a51  .  (5/32)a^ 
and  (9/32)  a^.  Since  the  a's  themselves  represent  6th  order  derivatives  of  rather  smooth 
tube  characteristics,  they  are  quite  small,  and  will  be  neglected  along  with  all  higher -  order 
terms  in  the  discussion  that  follows. 

If  it  is  assumed  that  the  a-^  and  terms  are  of  about  the  same  magnitude,  it  fol¬ 
lows  that  the  average  output  W  is  given  by 


W  =  E 


X-Y-  + 

1  1L 


Ta13XJi3  +4a31X 


N} 


18-2-1) 


From  this,  it  is  seen  that,  for  x  and  y  independent,  W  is  zero,  but  for  x  and  y  not  independ¬ 
ent.  the  average  output  of  the  multiplier  is  increased  by  the  distortion  terms  and,  in  particular, 

4 

if  x  =  y,  the  increase  is  about  d  *  x  .  The  d  used  here  is  the  measured  distortion  coefficient 
<a31  +  a13'  ' 

A  case  of  greater  interest,  however,  is  that  arising  when  y  is  the  sum  of  an  inde- 

Z  2 

pendent  noise  and  a  part  of  x.  Thus,  y  becomes  y  +■  ax.  Here  the  ratio  of  (ax)  to  Y  is  the 
familiar  S/N  or  p  . 

If  the  magnitudes  of  x  and  y  are  about  the  same,  as  is  desired  in  practical  cases, 
the  substitution  of  these  values  into  Eq.  (8-24)  (for  p«  1)  will  yield 

W  =  n7s  r2  [1  +  6d  e“2  (1  +4p)]  •  (8-25) 


—  2 

where  e  is' the  mean -square  grid  signal  voltage  applied  to  the  tube. 

S 

In  a  similar  manner,  involving  a  good  deal  of  algebra  that  is  omitted  here,  the  var¬ 
iance  of  W  is  found  to  be 
2  . 


VV 


N 


1  +  Ud  e  ( 1 
g 


3 

+  TP) 


(8-26) 


The  ratio  of  the  standard  deviation  to  average  output  in  the  absence  of  multiplier  dis¬ 
tortion  is  found  to  be  J N/ nS  or  (E  /N  )  2  .  Here,  it  is  seen  that  the  ratio  becomes 


/  °~  w  \  7aw\ 

\  W  /actual  \  W  / 


ideal 


}  +  ‘2d  eg2  (1  +  -|-P)]  2 


(8*27) 


2 

in  which  p«  1  and  e^  is  less  than  unity.  Under  these  conditions,  the  effect  of  multiplier  dis¬ 
tortion  is  kept  quite  small,  and  may  be  ignored  in  the  final  results,  at  least  where  the  assump¬ 
tion  of  reasonable  operating  point  and  attendant  value  of  d  about  0.01  is  valid. 
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CHAPTER  IX 

CONCLUSIONS 

The  results  thus  far  given  are  summarized  as  follows.  From  the  theoretical  point 
of  view,  it  has  been  shown  that  NOMAC  systems  are  members  of  a  class  of  optimum  communi¬ 
cation  systems  when  the  noise  is  considerably  larger  than  the  signal.  NOMAC  systems  deliver 
to  the  user  of  the  communicated  information  a  choice  made  by  the  receiver  from  the  set  of  pos¬ 
sible  transmitted  symbols.  The  class  is  optimum  in  that  the  symbol  thus  chosen  is,  with  prob¬ 
ability  approaching  unity,  the  most  probable  (a  posteriori)  of  the  symbols  that  might  have  been 
transmitted. 

The  fundamental  parameter  limiting  the  performance  of  these  systems  is  the  ratio 
of  signal  energy  to  noise  power  per  cycle.  Any  combination  of  bandwidth  ratio  (or  n  =  2TW)  and 
signal-to-noise  ratio  leading  to  the  same  ratio  Es/N^  is  capable  of  substantially  the  same  per¬ 
formance,  within  the  limits  of  signal-to-noise  ratio  less  than  unity  and  bandwidth  ratio  much 
greater  than  unity.  In  particular,  small  signal-to-noise  ratio  may  be  offset  with  large  band¬ 
width  ratio,  regardless  of  how  small  the  former  might  become. 

It  is  shown  that,  if  the  decision  made  by  the  receiver  is  based  on  maximum  correla¬ 
tion,  for  information  rates  not  exceeding  the  channel  capacity,  the  per-unit  equivocation  may  be 
reduced  to  any  desired  value  by  allowing  sufficient  delay  (coding  time). 

On  the  other  hand,  if  the  detection  criterion  is  that  of  exceeding  a  threshold,  an 
equivalent  loss  of  channel  capacity  is  occasioned  by  an  arbitrary  choice  of  the  threshold.  Even 
when  the  optimum  threshold  is  used,  a  delay  an  order  of  magnitude  greater  than  that  required 
when  the  criterion  of  detection  is  maximum  correlation  is  necessary  to  obtain  the  same  per-unit 
equivocation.  The  compensation  in  favor  of  threshold  detection  is  the  reduction  of  equipment 
complexity  in  the  receiver  performing  this  type  of  detection. 

An  important  disadvantage  is  connected  with  the  results  just  given.  This  disadvan¬ 
tage  is  the  stringent  requirements  of  synchronization  and  storage  associated  with  ideal  cross- 
correlation,  in  which  process  copies  of  each  of  the  possible  message  waveforms  are  available 
for  correlation  at  the  receiver.  If  it  is  assumed  that  storage  facilities  do  exist  at  the  receiver 
capable  of  storing  the  possible  signals,  the  ease  of  using  random  samples  of  noise  to  represent 
the  transmitted  symbols  is  lost,  and  the  problem  of  time  synchronization  to  enable  correlation 
to  be  performed  at  the  11  t  =  o"  point  of  the  correlation  curve  is  one  of  major  proportions. 

It  is  shown  that,  if  transmission  of  the  reference  copies  of  the  possible  signals  as 
well  as  the  intelligence  signal  is  resorted  to,  there  is  a  loss  of  effective  channel  capacity  (or 
effective  ratio  of  Es/Nq)  corresponding  to  the  ratio  of  signal  power  to  signal-plus-noise  power 
in  the  auxiliary  channel  or  channels.  This  use  of  auxiliary  channels  implies,  in  addition,  an 
expenditure  of  at  least  twice  the  bandwidth  used  by  stored  signal  systems.  To  its  advantage, 
the  so-called  ritwo-signal  system11  is  free  of  the  synchronization  problem,  and  makes  possible 
the  use  of  random  samples  of  currently  generated  noise  as  symbols. 

An  additional  apparent  disadvantage  is  the  " self-noise"  that  results  from  the  short- 
time  auto-correlation  of  noise  signals.  It  is  only  an  apparent  fault,  however,  in  that  this 
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component  of  noise  is  (1)  small  compared  to  other  noise  for  small  values  of  channel  signal-to- 
noise  ratio*  and  (2)  precisely  specified  by  the  knowledge  of  the  waveform  being  correlated  (which 
is  assumed)  and  thus  could  (with  sufficient  equipment  complexity)  be  eliminated. 

19 

One  version  of  the  stored  signal  type  of  NOMAC  system  has  been  suggested  by  Fano. 

It  is  a  variation  of  the  matched-filter  technique  of  North,  Middleton  and  VanVleck,  and  seems 
sufficiently  promising  to  merit  further  investigation.  This  system  apparently  solves  the  syn¬ 
chronization  and  storage  problems  simultaneously  at  the  expense  of  somewhat  greater  per-unit 

19 

equivocation.  A  complete  introduction  to  the  method  is  found  in  the  reference  cited. 

From  the  practical  point  of  view,  it  is  demonstrated  that  laboratory  models  of 
NOMAC  are  easily  realized  in  comparatively  compact  equipment.  This  is  partly  a  result  of  the 
use  of  converter  tubes  as  multipliers,  which  are  shown  to  contribute  insignificant  error  when 
properly  used.  Also,  the  use  of  simple  filters  as  integrators  is  shown  to  be  feasible.  While 
the  per-unit  equivocation  is  increased  by  the  reduction  in  effective  integration  time  (delay)  when 
filters  are  used  to  perform  integration,  it  is  felt  that  the  disadvantage  is  compensated  by  sim¬ 
plicity  of  equipment. 

Much  remains  to  be  determined  about  the  properties  of  NOMAC  and  related  commun¬ 
ication  systems.  It  is  recalled  that  all  the  results  reported  here,  theoretical  and  experimental 
(as  far  as  practicable),  are  given  for  but  one  type  of  channel  disturbance  -  that  of  additive  white 
Gaussian  noise.  How  these  systems  will  behave  in  the  presence  of  other  noise  effects,  or  when 
they  are  subject  to  multipath  propagation,  remains  an  important  problem  for  further  investiga¬ 
tion. 

Also,  a  detailed  study  of  combinations  of  various  modulation  methods  and  random 
carrier  signals  is  recommended.  For  example,  a  frequency-modulated  random  carrier  has 
been  tried  in  the  laboratory  with  considerable  success,  and  obviously  merits  extensive  investi¬ 
gation  to  determine  its  applicability  to  secure  voice-communication  links. 


^Suggested  by  R.  M.  Fano,  11  June  1951. 
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APPENDIX  I 


PER-UNIT  EQUIVOCATION  AND  PROBABILITY  OF  ERROR 

If  Hx  represents  the  entropy  of  the  information  source  and  is  the  equivocation 

or  remaining  uncertainty  inherent  in  the  channel,  the  per-unit  equivocation  is  defined  as 


H 


P.U.E.  = 


■/z 


H 


(A  1  - 1 ) 


For  systems  of  the  type  considered  here,  =  log  (K  +  1).  The  equivocation  is 


given  (see  Ref*  5)  by 


Hx/z  =  “  I  plz>  I  P<x/z)  lo&  P<x/z>  • 


(Al-2) 


Expression  (Al-2)  is  interpreted  to  mean  the  average  over-all  received  signals  of  the  uncer¬ 
tainty  represented  by  the  a  posteriori  probability  distribution  following  reception  of  a  signal. 
For  the  purposes  of  calculation,  the  a  priori  probability  of  any  one  of  the  possible  symbols  is 
taken  as  l/K  +  1.  The  a  posteriori  probability  of  the  indicated  symbol  is  ( 1  -  p),  where  p  is  the 
probability  of  error,  and  the  a  posteriori  probability  for  each  of  the  remaining  K  symbols  is 
taken  as  p/K.  The  equivocation  then  becomes 

Hx/z  “  “  ft1  "  P)  l°S  (1  -  P)  +  P  log  FI  .  ( A 1  -  3 ) 


Fig*  A  1.1.  Per-unit  equivocation  vs.  probability  of  error. 
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The  resulting  expression  for  the  per-unit  equivocation,  which  is  the  upper  limit  that 
can  be  associated  with  any  given  probability  of  error,  is  then 


P.U.E. 


(p  -  1)  log  (1  -  p)  -  plog  £■ 


log  K  +  1 


(Al-4) 


follows. 


An  expression  for  P.U.E.  when  p  becomes  very  small  (the  practical  case)  is  as 


P.U.E.  =  p  \ 


log  e  +  log  ^ 


log  K  +  1 


(Al-5a) 


and  K. 


Plotted  in  Fig.  Al.l  and  tabulated  below  are  typical  relations  between  P.U.E.,  p, 


TABLE  AI-  1 


PER-UNIT  EQUIVOCATION 

Corresponding  To  Probability  Of  Error  When  Receiver  Chooses 
Most  Probable  Message  As  Message  Transmitted 


58 

UNCLASSIFIED 


UNCLASSIFIED 


APPENDIX  II 


THE  DISTRIBUTION  OF  VECTOR  MAGNITUDES 


Given:  the  probability  density  distribution  for  x,  namely* 


p(x)  =  —  exp 

\TZv  S 


2' 

X _ 

7  2s_  ’ 


Now  [x[^  =  £  x7,  so  that 
i  1 


r  21 

x. 

L  2s 


p(^,. 

l4  x  2  *>/  2ttS 


(A2- 1) 


(A2-2) 


and,  where  <p  (t)  is  the  characteristic  function  related  to  the  distribution. 


1>‘  (t)  =  f  exp[itx2)  p(x2)  dx2 
v-  00  L 


(A2-3) 


[1  -  2iSt]~  2  , 


(A2-4) 


*0'(t)  =  [0'  (t)]n  =  [1  -2iSt] 


(A2-5) 


P0'(|X|2)  =  -~f  exp  [—it  |  X  |  2]  0Q'(t)  dt 

J-  oo  r  .  .  fl 


podx|)=  Aix 


°i  r  ixi2 

n-l  exP|_ - 2§d 


(2S)2  ^(f) 


(A2-6) 


(A2-7) 


The  distribution  (A2-7)  may  be  derived  in  another  way  which  reveals  significantly 
what  is  involved  in  p  ( I X  ( )  . 

First,  the  probability  density  distribution  for  a  given  point  in  n- space  is  given  by 
the  product  of  the  probability  density  distribution  for  each  of  the  Cartesian  coordinates  {the  co¬ 
ordinates  are  independent).  Thus 


P"(X) «  p(xA)  p(x2) . p(Xn) 


(A2-8) 


■  (A>'exp[-i(xi2+x2Z*  --  x„2il  ■ 


(A2-9) 


•  <is>2  exp[-is'(lxl2)| 


(A2- 10) 
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In  (A2-10),  ]X|  is  the  squared  magnitude  of  the  distance  of  a  point  X  from  the  or¬ 

igin,  and  the  probability  density  of  the  point  is  seen  to  depend  only  on  this  distance. 

The  probability  density  distribution  for  the  distance  |x|  is  the  integral  of  p"  (X) 
taken  over  all  points  that  lie  distance  |x|  from  the  origin.  Since  |  x|  is  constant  in  the  inte¬ 
gration,  pQ  C f  X |  )  is  the  surface  area  (n  -  1  space)  of  the  hypersphere  of  radius  [  X[  times  p"(X). 
The  surface  area  is  in  turn  the  rate  of  change  of  the  volume  of  the  hypersphere  with  respect  to 
the  radius  at  that  value  of  radius. 

The  volume  of  a  hypersphere  in  n-space  can  be  obtained  as  follows. 


r  n  /lx  I  pJ I  xl  2-d  2  f\l I  xl  z-x  d  2 

r»<lx[)vO=2/  *f«4  <  'J0  i=1  «» 


(A2-11) 


<r  2  =  lx  I  2  —  2  <  2  , 

i  r:  1  1 


2  =  I  xl  2 


vn(|xl )  .  t'J**  d<, f-1  d<  , 

V  0  •'O 

V„  11*1 )  '  d<j  i/”-1  d<jj  . .  ,/2  J7*  -  dl{  , 

47  O  uO  VQ 

But  i 

-c.f 

Z 

is  recognized  as  the  Beta  Function  (1/2  cr^  )  P(l/2  »  3/2).  Generally 

r  <aZ  Z^Z  H  1  q  + 

J  (Q  "  P  )  dp  =  ya  P(j.  V>  ' 


Therefore, 


1  r  3 


(A2-12) 


(A2-13) 


Vn(  !X  !)  =  2"~r -J^TT^/0  n  A^JQ  ‘  '  JQ  r+2  (Vn)  d^n-r»l  ■  <A2-‘ 


so  that  when  carried  out  to  r=  n  —  1 


1  n  1  ? 

r<i>  r(fJ  n 


v  (  X  )  =  2  i — - “ 

n  r^p)  n 


(A2-15) 
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_  |  x  [n 

The  surface  S  (|  Xj )  of  the  hypersphere  is  thus  readily  obtained  as 


sn(|x|) 


n/2  I v” I  n  1 


=  2n  '  lx] 


nf) 


(A2-16) 


(A2-17) 


and  the  product  of  (A2-10)  and  (A2-17)  is 

.  2TTT|x|n~1  ^ 

ref) 


P0(lx|) 


(2ttS)-^- 


.  AM 


n-1  exp 


I  !x| 2] 
L  2S  J 


(2S)n/2  P f) 


(A2-18) 


(A2-7) 


Some  of  the  properties  of  this  distribution  are  as  follows. 

n+1 


n  r 

Ixl2'" 

2  Ixl  n  6XP  t 

2S  J 

(2S)n^  nf) 


d  lx  |  = - -L-  - 

(2s)fnf) 


(zs) 2  r 


^ns 


E 


/„ 


r@ 

lim  E  [[xj]  =  \fnS~  . 

n-*oo  ^ 

2  |X  I n+ 1  exP 


(2S)n/2  T(f) 


dpi  = 


(2S)n/2r  (f) 


f +  1 

(2S)2  T(t  +  1) 


=  nS 


«T|2  |  =  nS 


r(- 


n  +  1\ 


nf) 


■nT2S 


(A2-19) 

(A2-20) 

(A2-21) 

,  (A2-22) 

(A2-22a) 

(A2-23) 


If  Stirling's  approximation  is  used  for  I^(n)  and  one  examines  p  ( |x|/V  nS),  there 


results  the  expression 


-A 


\n-l 


V  nS  / 


exp  <— 


tS-i 

*/nS 


(A2-24) 
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which  is  valid  for  n  of  the  order  of  ten  or  higher. 

One  notes  that,  for  |X|  =  \f  nS  ,  the  value  of  the  density  distribution  function  is  \f  n/V 
and  that  the  variance  of  the  distribution  behaves  as  l/n  .  Therefore,  the  distribution  function 
behaves  somewhat  like  a  unit  impulse  function  at  |  X|  / sf  nS  =  1  .  The  distribution  is  given  for  a 
few  values  of  n,  tabulated  in  Table  AII-1,  and  plotted  in  Fig.A2.1. 


TABLE  All- 1 


NORMALIZED  PROBABILITY  DENSITY  OF  VECTOR  MESSAGE 


n  =  3 


(piV 

VnS/ 

p  (  MN 
°\nTSs~, 

0.01 

0.043 

0.25 

0.805 

0.667 

1.070 

1.000 

0.977 

2.000 

0.438 

4.000 

0.043 

n  =  10 


( ixiV 

p  flxl> 

VnTSs ) 

°vn$, ) 

0.50 

0.965 

0.90 

1.830 

1.00 

1.783 

1 .20 

1.490 

2.00 

0.271 

n  =  100 


/ix|Y 

Y/~nS7 

p  ^ 
°WT37 

0.81 

2.26 

0.90 

4.52 

0.96 

5.52 

0.99 

5.66 

1,04 

5.35 

1.10 

4.29 

1.21 

1.96 

0.4  0.8  1.2  1.6  2.0  2,4 

(|X  I /VnS) 


Fig.  ALL  Probability  distribution  of 
relative  amplitudes  of  vectors. 
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APPENDIX  III 


PROBABILITY  OF  ERROR:  ADAPTATION  FROM  S.  O.  RICE 


Equation  (4-12)  of  S.  O*  Rice's  article  "Communication  in  the  Presence  of  Noise  - 
Probability  of  Error  for  Two  Encoding  Schemes"  is  given  (in  the  nomenclature  of  this  paper) 
as 

P(no  error)  =  P(Dj,  D^,  D^,  •  •  •  D^>Do) 

(A3  -  I) 

=  /^°°d|z|  f”d\Y\  p(|Z|.  |Y|)  [l  -  P(|Y|.  |Z|]]K 
o  o  o 


In  his  Appendix  I*  Rice  gives  as  an  approximation  for  P(]Y| ,  |z|)  the  expression 

2 


p(|Y|,|Z|)  -  exp 


_ &L- 


2  o* 


D 


( A3-2) 


where  -  E  [p  -  E)J  and  p  ~  D  -  ) Y]  .  This  expresses  the  probability  that  any  one  dis¬ 

tance  ("D”  above)  from  the  point  Z  to  the  point  be  less  than  the  magnitude  of  the  noise  vec¬ 
tor  Y,  under  conditions  of  fixed  |Z|  and  fixed  |Y| .  The  result  is  then  averaged  over  all  mag¬ 
nitudes  of  2  and  Y. 

An  approximation  for  the  probability  of  error  results  by  taking 
1  -  P(Dj,  D^,  D y  •  •  ■  D^>Dq)  which  for  small  P  (error)  is  given  approximately  by 


P (error)  =  K  f*  d\Z\  /*“  d |y|  p(|Z|,  |Y|)  P(|Y|,  |Z|) 
U  o  t>  o 


(A3-  3) 


As  n  increases,  p(  |z| ,  |Y| )  approaches  an  impulse  with  all  significant  contribution 
to  the  integral  in  the  vicinity  of  |Z|  =  nP  and  |Y|  =  nN.  Evaluating  the  integral  at  these  values 
of  |Z|  and  |Y|  yields  a  rough  approximation  for  the  probability  of  error,  namely, 


P (error)  ^  exp 


_2 
_ 

L  zWD  J 


(A3-4) 


The  variance  defined  earlier  as  the  mean  square  difference  of  the  distances  and  their 

mean  is  given  by  Rice  in  terms  of  |z| ,  but  when  evaluated  at  the  average  \Z\  equals 
2nS  (  S  +  2P).  Rice  gives  p  in  terms  of  |Y| .  Again,  if  evaluated  at  the  average  |Y| ,  the  result 
is  /u  =  D  —  |Y|  =  n(P  +  S  -  N)  -  2nS.  For  these  values,  the  approximation  reads 


P (error) 


K 


r  [FW)-  [-s¥ip]  • 


Where  S«N,  this  may  be  written  even  more  approximately  as 

iv  yft 


P (error) 


K 


N 


Y  nS 


exp 


nS 

2N 


] 


(A3-5) 


(A3-6) 
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This  expression  can  now  be  written  in  terms  of  the  approximations  nS/2N  =  C'T 
for  S  «  N  and  K  =  expfH’T]  for  K  »  1  .  The  result  is  then 

P(error) -x- - -==  exp  (Lc'T(  1  - -£) 1  .  (A3 -7) 

L  ^  J 

and  is  reasonably  valid  for  very  large  n,  large  K,  and  large  noise -to-signal  ratios.  The  first 
two  conditions  are  the  same  as  those  introduced  by  Rice  for  the  validity  of  his  probability -of -no- 
error  expression.  The  third  is  added  to  show  the  similarity  of  his  results  to  those  of  Chapter  IV. 
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APPENDIX  IV 

THE  ARBITRARY  ORIENTATION  OF  THE  VECTOR  FIELD 

Let  the  set  of  possible  message  vectors  be  set  up  in  a  matrix  [Sk  +  1 ,  n]  .  The  sub¬ 
scripts  indicate  rows  and  columns  in  that  order,  where  the  vectors  are  arrayed  by  rows,  their 
coordinate  components  by  columns. 


X01  X0 2 


X11  X12 


Sk+l.n  = 


(A4-1) 


It  is  assumed  that  this  matrix  is  known  to  both  the  transmitter  and  receiver,  and  the 
transmission  of  a  message  is  represented  mathematically  as  the  selection  of  one  of  the  K  +  1  rows 
(arbitrarily,  the  vector),  and  adding  another  row  matrix  to  this  row  and  delivering  the  re¬ 
sult  to  the  receiver.  The  resulting  row  matrix  is  ,XA .+  .Y.-.X.,  +  Y  .  The  transpose  of  this  matrix 

°  UU  L_r  L  0  i  n 

is  designated  by  Zj  ,  and  at  the  receiver  the  operation  of  +  j  J  post-multiplied  by  ZJ  is  per¬ 
formed.  The  resulting  column  matrix  of  K  +  1  rows  is  designated  by  wj .  The  receiver  chooses 
which  of  the  vectors  in  the  M  matrix  was  most  probably  transmitted  according  to  the  values  of 
the  elements  of  W.  The  operation  is 


fst  ,  .  I  Z  1  =  [w,  .  ,  ,1  =  W* 

I  k+  1,  n  n  k  +  1,  1 


(A4-2) 


Next,  one  takes  a  unit  orthogonal  transformation  J  such  that  the  vector  is 
transformed,  for  example,  to  lie  along  one  of  the  coordinate  axes,  arbitrarily  the  first.  Then 


Sk+  l,.nj  £An,  n  =  Sk+l,nJ 
In  this  transformation,  it  is  apparent  that 


(A4-3) 


2,  Xoi^j  *  N 
1=1  J 


s  X^a  =0,  j  *  1 

1=1  J 


Further,  the  sum 


Z  a  .  a  . 

.  .  ri  si 

i=  1 


where  6r  is  the  Kronecker  delta.  The  set  a,  of  elements  will  be  an .  =  Xn./jXn|  while  the  values 
s  i  H  Ui  U 

of  the  other  elements  of  A  are  determined  by  the  orientation  of  the  vector  field  about  the  first 
coordinate  axis,  which  is  arbitrary. 
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Obviously 


k  +  1 ,  n 


in  which 


But 


A 


n,  n 


1 

— 

~  — 

-1 

— 

A 

A 

Z 

= 

W.  +  1 ,  1 

n,  n 

n,  n 

k 

i" 

-1 


n,  n 


(A4-4) 


(A4-5) 


- 

A 

Z 

n,  n 

t 

— 

|Xr 


'»•»],  H, 


(A4-6) 


Since  Y  is  chosen  at  random  in  the  n-space  in  the  first  place,  it  might  well  be  chosen 
after  orientation,  and  thus  one  may  write  Y J  for  J  JyJ  .  Then,  the  final  matrix 


✓* 

]x0r 

0 

V 

Y2 

> 

+ 

.  ! 

! 

0 

c1 

•  >* 

_ i 

J 

(A4-7) 


A  closer  inspection  of  the  structure  of 


e  i 

k  +  l,n_ 


reveals  the  matrix  below. 


j^Sk  +  1 ,  nj 


Xr 


0  .  . 


xh  xi  2* 


0 

X 


In 


x*,  X'  .  .  .  x; 

kl  k2  kn 


(A4-8) 


The  X!  /s  are  also  randomly  oriented  in  n-space  so  that  they  fit  the  same  probability  density 
distribution  after  coordinate  rotation  as  before. 

In  a  similar  manner,  one  might  choose  J^An  J  to  reorient  the  vector  field  so  that  5? 
falls  along  the  first  coordinate.  In  this  instance,  the  combination 


A  1  Z  1 
_  n,  nj  t  nj 


|Z| 

0 


(A4-9) 
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while  the  combination 


A 

:+  1 ,  n  n,  n 


.  xv 


(A4-10) 


[Xkl  Xk2  •  -  •  XknJ 

(Again  the  numbers  XjV  for  i  ^4  0  are  from  the  same  density  distribution  after  coordinate  rotation 
as  before.  )  The  result  finally  obtained  is  given  by 


k  +  1 ,  1 


(A4-1X) 


The  result  indicated  by  the  matrix  wj  is  the  same  as  that  indicated  by  the  column  matrix  of  com¬ 
ponents  of  the  vectors  in  the  2?  direction,  and  thus  a  decision  based  on  the  values  of  these  compo¬ 
nents  is  equivalent  to  one  based  on  the  elements  of  w]  . 
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APPENDIX  V 

THE  PROBABILITY  DENSITY  FUNCTION  FOR  CORRECT  OUTPUTS 

The  correct  correlation  output  for  any  of  the  systems  described  in  which  a  noise- 
free  version  of  the  reference  signal  or  signals  is  available  to  the  receiver  has,  by  arbitrary  la¬ 
beling  of  the  results,  been  designated  Wq  .  This  value  is  given  by 


Wo=lX0l  +VY  ’ 


(A5-1) 


W n  =  |XJ  lIXl  +  Y.]  . 


(A5-2) 


where  the  methods  of  Appendix  IV  have  been  used  to  reduce  the  dot  product  of  Eq.  (A5-1)  to  the 
single  algebraic  product  in  Eq.  (A5-2). 

The  probability  density  distribution  function  of  outputs  Wq  is  designated 
and  can  be  expressed  by 


exp  ~ 


(w0  - 1 xQn 


r"  2N  |x  | 

p2(W)  =  /  p  (|x  |)  d|x  |  - f  _  r  . — r 

L  0  Jo  °  °  °  >T2i5 T-  X 


(A5-3) 


The  exponential  expression  of  Eq.  (A5~3)  is  simply  the  Gaussian  distribution  function  with  aver- 
2  2 

age  |XQI  and  variance  N|Xq|  .  Thus,  ii  is  the  conditional  probability  function  for  the  value 

|X  |  2  +  |X  |  Y ,  in  Eq.(A5-2)  subject  to  given  (fixed)  |X  |  .  In  Eq.  (A5-3),  p7(W  )  is  obtained 
o  o  i  o  £  o 

by  averaging  the  conditional  distribution  over  all  | |  . 

If  the  expression  for  (  |Xq  | )  is  inserted  in  Eq.  (A5-2),  it  becomes 


P2(Wo> 


^|Xoir  1  exp 


|X  \Z 

1  O  1 

r  2s  _ 


exp  - 


(w0  ~  iy_) 

2N  |xl2 


(zsf/z  rtf) 


rT^W  |XQ| 


(A5-3a) 


which  can  be  simplified  somewhat  by  letting  |Xq  |  =  a  .  Then 


n+3  (W  -  a) 

— ^ —  a  1  o  / 

OO  2  exp  -  ?c  -  ?  Mr, 


pCO 

P2(Wo)=/  A 
U  o 


•/2¥IT(2S)n/2  nf> 


(A5-3b) 


The  expression  given  by  Eq,  (A5-3b)  can  be  integrated  into  a  complicated  expression 
involving  modified  Bessel  functions  of  the  second  kind.  However,  the  characteristic  function  is 
more  easily  obtained  and  can  give  a  great  deal  of  information  about  the  probability  distribution 
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in  an  easy  manner.  Thus 


(t)  exp[jW,t]p?(WJdWrt  , 


1  J  Hi  l  T*  ;  ■ 

o  J  ^2  '  o  o 


exp 


(Wo-  |Xo|  ) 


n°°  r°°  I  2N  |X„r 

sf  Po(|Xo|j  d|Xo!/  exPLWot]  - 

yj  -  00  -  OO 


dW 


TW|xJ 


Let  (W  -  |X  |2)  =  v  ,  then  dW  =  dv  ;  W  =  v  +  |Xj2  . 
'  o  1  o 1  o  o  o 


<PZ  (t)=r  po(|XQ|)  d|x0|  exp[ j  |Xo  1 2  t]  f  exp[ jvt] 
Jo  J  -OO 


exp 


v2 


2N  X 


dv 


yfZvW  [X. 


=  [  Po(lXol]  dlXol  exp[j|Xo|Zt]  exp 
Jo 

£°  2  lxp In~  1  exP  t~lX0lZ  ^Is  + 


N|xor  t 


(zs)n/2  ref) 


Nt 

— -Jt}_ 


d  X 


The  last  expression  can  be  easily  integrated.  If  a  is  again  substituted  for  |X 


(t) 


1 

oo  Z  exp 


■/  - 


,  1  .  Nt2  ... 


(2S)f  rtf) 


da  , 


so  that 


<t>i  (t)  =  TJz - r~ 

(2S)n/^  rtf) 


rtf) 


,  1  Nt  ,.n/2 

(Is  +  —  -Jt) 


=  [  1  +  SNt  -  2jSt] 


n_ 

Z 


From  this  characteristic  function,  one  may  find  the  moments,  and  from 
a  reasonable  approximation  to  p,  (W  )  in  normal  form.  For  example, 


W 


1  d 


n+Z 

2 


o  j  dt 


*2(t)lt=o  =  (rf)  11  +  SNt  ~  2jS‘]  (2SNt-2jS)|t=o 
=  nS  . 
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(A5-4) 

(A5-4a) 

(AS- 5) 

(AS- 5a) 

(A5-5b) 

(AS-  6) 

(A5-  6a) 

(AS-  7) 

them  obtain 

(AS-  8) 
(A5-8a) 
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_  n+2 

Wo2=  -TiTtCl+^-ajSt)  2  (jnS-nSNt)]  |t=o 

f  n+4 


(A5-9) 


=  -|(-^>  (1  +  smz  -  2jSt)  2  (-2n)  (SNt  -  jS) 
n-2 

4(14  SNt2  -  2j,S)  2  (— nSN) 

=  nS2  (n  +  2)  +  nSN 

From  the  knowledge  of  Wq  and  ,  the  variance  can  be  computed  as 

a^j.  =  nS2  (n  4  2)  4  nSN  -  (nS)2  , 

o 

c =  nS  (N  4  2S)  . 

O 


(A5-9a) 

(A5“9b) 


(A3-  10) 

(A5-10a) 


Therefore,  in  as  much  as  (Wq)  is  the  average  of  a  Gaussian  distribution  with  varying  average 
and  variance,  one  may  approximate  it  with  the  normal  density  distribution  function, 


P2  (Wo) 


1 

- exp 

V  2irnS  (N  I-  2S) 


(Wo-nS)2 
2nS  (N  4  2S) 


(A5-11) 


This  expression  is  valid  chiefly  over  the  bell-shaped  part  of  the  curve,  and  a  more  precise  know! 

edge  of  p,  (W  )  is  required  for  an  accurate  knowledge  of  the  function  along  its  skirts.  Returning 
£  o 

to  Eq.  (A5-3b)  and  rewriting  it  slightly,  one  has 


P2CWq)  =i 


n 

r  a. 

^3. 

2  exp 

f 

o 

Wl 

exp 

.N  J  , 

W  2  -  2W  Q  4  a2 
o  o 


r  2s 


2Na 


(2  s)n/2  F(S) 


-da 


czsr'*  n?> 


n-3 

f  2 

—  a  exp 

n)Jo 


W 

11.  o 

~n  '2S  2N'  "  2Na 


da 


(A5-  12) 


(A5-12a) 


To  evaluate  the  last  given  expression,  reference  to  G.  N.  Watson's  "Theory  of  Bessel 
Functions"  (p*  18  3,  formula  15)  reveals 


exp  -t  -  - 


4t 


dt  = 


2Kv  < Z ) 


(A5- 13) 


(fZ)  v 

To  put  the  integral  of  Eq.  (A3- 12a)  into  the  same  form,  a  substitution  of  t  for  a(P/2SN)  yields 
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__  n-3 

r  2 

I  a  exp 

J  O 


,  P  1 

“Lsn*  “ 


w 


.  NS  P 

4  -  ■  ...  Q- 


P  2SN 


da 


l 


n-3  , 

OO  2 

,2SN»  .  2 

(-p~)  t  exp 

-^y  -  1-n  . 

.2SN.  2  P°°  2  ** 

=  He~)  J  t  exp 


-t  -  — ^ 

4N  St 


dt 


-t  - 


Thus,  in  terms  of  Eq.  (A5-13),  v  =  (1  -  n)/2  ,  and  Z  is  (W  /N)  P/S  *  Therefore, 


_  n-3 

r  2 

j  a  exp 
•J  o 


„  W 
P  o 


2SN  4Na 


PW  2\ 

O 

|  1 

dt 

N2s  ) 

)  41 

-  n)/2  , 

,  and  Z 

n-1 

■  (2®N ) 

2 

2K 

<- 

n-1 

/E  ) 

2  \  N  V  S  / 


1-n 


IE} 

(2  N  V  S  ; 


»(¥> 2  isli 2 

?.  \  > 


n-1 


n-1 


AV 


o  /P 


=  2f)  K-1I»JS 


Therefore, 


P2  <Wo> 


- 

W  j 

2  exp 

_  — 9_  2 

.  N  Jo 

(2sjn/2  nf) 


,e”;1  k  ft.  £ 

(S }  n- 1  N  VS 


This  expression  may  be  used  in  connection  with  the  limited  tables  of  the 
tion  if  high  accuracy  is  a  requirement. 
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(A5-14) 


(A5-  14a) 


(A5-15) 


,  (A5-  1 5a) 


(A5-  1 5b) 


(A5-16) 

Bessel  Func- 
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INTEGRATION  TO  OBTAIN  THE  PROBABILITY  OF  ERROR 


In  Chapter  VII,  the  expression  for  the  probability  of  error  applicable  to  the  system 
investigated  experimentally  [see  Eq.  (7~7)J  is 


P  (error)  - 


exp  [—  ~(a^  —  1)]  erf 


(A6-1) 


To  obtain  satisfactory  values  of  P  (error),  this  expression  has  been  integrated  nu¬ 
merically  by  methods  designed  to  give  results  within  about  one  per  cent  of  the  true  value  of  the 
integral.  Such  integration  is  seen  to  be  necessary  for  small  values  of  n,  since  the  changes  in 
erf  n/  n  p/4a  are  great  over  the  range  of  values  for  which  PQ(a)  is  substantially  different  from 
zero  (see  Appendix  II). 

The  integrand  of  (A6-1)  is  seen  to  vanish  faster  than  a  simple  exponential  for  largL 
a  ,  and  it  may  be  shown  that  the  integrand  for  very  small  a  is  less  than  exp  [-n/2  (1  —  a)4]  . 

Thus,  it  is  sufficient  for  the  required  accuracy  to  integrate  only  over  those  values  a  for  which 
the  integrand  exceeds  exp  [—5]  times  its  value  at  a  =  1  .  The  value  at  a  =  1  is  not  the  maximum 
value  of  the  integrand,  but  is  of  the  same  order  of  magnitude. 

To  determine  the  limits  of  a  over  which  integration  should  be  extended,  an  approx¬ 
imate  value  for  the  integral  (A6-1)  is  used.  Where  f(n,p)  is  used  for  the  probability  of  error 
for  fixed  values  of  n  and  p  ,  the  new  form  is 


f  (n.  p) 


[Z~ fa  r  2.,  / 1  +  a  r  np 

■JtJ  1  /¥  exP  [~na  wsr  exP  t-  srrr 


The  ratio  of  the  integrand  to  its  value  at  a  =  0  ,  (a  =  1)  ,  is  given  by 


ratio  =  1  +  a  exp  ~n  -  8fl^+”aj) 


(A  6  -  2 ) 


(A6-3) 


It  is  desired  to  determine  the  values  for  which 


8(1  +  a)  '  n 


( A  6 -4) 


It  is  obvious  that  for  positive  a,  a /(I  +  a)<  1  ,  so  that  if  a  =  (5/n)+(p/8)  the  inequal¬ 
ity  is  satisfied.  Furthermore,  if  by  this  equation  is  less  than  one,  a /(I  -i  a)  does  not  exceed 
one-half,  and  the  expression  for  positive  a  (or  upper  limit  of  integration)  becomes 


/JL  +  JL_ 

In  16 


(A6-5) 


The  argument  leading  to  Eq.  (A6-5)  is  not  valid  for  a  negative.  Consider  (3  =  -a  ,  and  for 
0  <  J3  <  1  it  is  desired  that 
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p2  + 


_&b_ 


8(1  -P)  n  ' 

For  small  p,  a  solution  to  the  inequality  is 
40 


P  = 


np 


(A6-6) 


(A6-7) 


Since  the  economy  of  knowing  (3  occurs  only  when  p  is  small,  nothing  more  need  be  derived 
about  p. 

From  Eqs.  (A6-5)  and  (A6-7),  one  may  write 


P  (error)  4  ~zjvf  2  &n  *  exp[-y(a2  -  1)]  erf  ^-j^da 


(A6-8) 


in  which  is  the  larger  of  zero  of  1  -  40/np  and  a^  is  given  by  1  +  ^{5/n)+(p/16 ) unless 
exceeds  two  by  this  formula,  in  which  case  1  +  *s/(5/n)+(p/8) should  be  used. 

Below  in  Table  A6-1  are  tabulated  results  of  this  numerical  integration  over  the 
limits  indicated  above.  The  probability  of  error  is  plotted  as  a  function  of  signal-to-noise- 
power  ratio  and  time-bandwidth  product  (or  dimension)  n  in  Chapter  VII  (see  Fig.  7.2). 


TABLE  AIV-1 

PROBABILITY  OF  ERROR  FOR  THE  BINARY  CHOICE  SYSTEM 
OBTAINED  BY  NUMERICAL  INTEGRATION 


n 

4- 

X^P^O.  001 

0.01 

0.1 

1.  0 

10.  0 

10 

0.480* 

0.433 

0.  307 

0. 0548 

1.  94  X  10“6 

10 

0.437* 

0.308 

0.0568 

3. 09  x  10"? 

-56* 

1.  30  X  10 

10 

0.309* 

0. 0566 

2,59  x  10‘? 

.66# 

1.  30  x  10 

0 

10 

0.  0569* 

2.  87  x  10“7 

1.  30  x  10 

0 

0 

10 

t.  88  x  10‘7* 

_  66  # 

L  30  x  10 

0 

0 

0 

*  =  entries  taken  from  the  error  function  table  of  1/2  erf \T np/4 
in  which  n  is  considered  sufficiently  large  that  the  change  in  value 
of  the  error  function  over  the  range  of  contribution  of  a  is  small. 

The  0's  are  inserted  in  those  positions  of  the  table  where  the  prob¬ 
ability  of  error  is  insignificant  in  the  lifetime  of  an  equipment. 
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APPENDIX  VII 

SCHEMATICS  OF  THE  EXPERIMENTAL  NOMAC  SYSTEM 


The  block  diagram  below  (Fig.  A7.1)  is  a  guide  to  the  complete  schematic  diagrams 
on  the  following  pages. 


Fig.  A7.1.  Block  diagram  of  experimental  system. 
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Schematic  diagram  of  random  -  signal  generator. 
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Fig.  A7.3/  Schematic  diagram  of  side-band  generator  (transmitter). 
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Fig.  A7.6,  Schematic  diagram  of  integrating  filter  (receiver). 
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APPENDIX  VIII 

THE  DISTRIBUTION  OF  SUMS  OF  PRODUCTS 

Although  the  probability  density  distribution  discussed  in  this  Appendix  does  not  ap¬ 
pear  in  the  main  body  of  the  paper,  it  was  extensively  investigated  during  the  preliminary  re¬ 
search  work.  Since  it  is  inherently  related  to  the  correlation  process,  a  review  of  the  results 
is  given  here. 

Let  TJ  be  defined  as  the  sum  of  products  XY  where  X  and  Y  are  independent  nor¬ 
mal  variates,  the  variance  of  the  former  being  S  and  that  of  the  latter  being  N  .  Thus, 


U  =  £  U.  =  2  X^. 
i=  1  i=  1  11 


(A8-1) 


The  joint  distribution  of  X.  ^nd  Y.  is  written  f^om  a  knowledge  of  their  individual  distributions. 


P  (X,  Y)  =  . 


Y* 

2N_ 


2"  sTSN 


(A8-2) 


The  probability  density  distribution  for  IK  =  X.Y.  is  given  by 


(|U  [)=T  — -Lr-exp  -Is 

1  J  o  TT's/sir a  ^ 


Z  U. 

a  i 


(A8-3) 


where  the  integration  is  carried  out  only  in  one  quadrant  of  the  joint  distribution,  which  is  sym¬ 
metrical  about  both  X  and  Y  axes.  As  given  in  (A8-3)  p'  is  an  even  function  of  on  the  left, 
and,  since  positive  or  negative  products  occur  with  equal  likelihood,  it  is  apparent  that 


pOJ*) 


tta/SNJo 


7  Z  U 

I  a  i 

—  exp  - 


da 


(A8-4) 


To  obtain  the  desired  density  distribution  function,  the  characteristic  function  is 
found  as  an  intermediate  step. 


0' (t)=f  exp[itU.]f  - 1 - exp  - 

J  -oo  U  o  tra  V  SN 


1 

L2S  J  | 


2  U 
a  i 

25  ~  ZNa2 


r  da  dU.  , 


=  /  da - - — -I  exp  - y  +  itU,  dU.  , 

Jo  J-  OO  2Na  1  1 


n  w  p  a  I  ? 

=Jo  7^exp  — r(s  +  Nt)  da 


( A  8  -  4 ) 


(A8-4a) 


(A8-4b) 
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=  [1  +  SNt2]"*  .  (A8-5) 

The  characteristic  function  of  the  desired  probability  density  distribution  function  p(U)  is  then 
given  by 


0  (t)  -  [  1  +  SNt 


(A8-6) 


and  from  it, 


oo  11 

p(U)  =  2^f  exP[~itu]  D  +  SNt2]  4  dt 

\J-  CO 


(A8-7) 


From  the  characteristic  function,  one  obtains  a  complete  description  of  the  distri¬ 
butions  of  U’s  .  For  example,  from  the  expansion  of  <p  (t) ,  one  obtains  =  nSN  and  thus, 
cry  =  nSN ,  since  U  is  zero. 

Foster  and  Campbell^  list  the  integral  (A8-7)  in  their  comprehensive  table  of  trans¬ 
forms.  In  particular,  transfer  pair  No.  569  links  l/[{5  +  p  ]a  and 

|g|°~l  Ka-  j(gjg|) 

'sTirr(a)  (2p)a-z 

When  this  is  applied  to  Eq.(A8-7),  the  result  is 


p(U)  = 


U  \  -  n  _  j  /  U 
2\TSN/  K  2  \  sTsfi 

n/^sn  r(f) 


(A8-8) 


The  K  (z)  here  is  the  modified  Bessel  Function  of  the  second  kind  for  imaginary  arguments  of 

v  13 

order  v  .  Many  sources  (e.g.r  Hildebrand  )  give  the  behavior  of 

Vz)^  2v_1r(v)rv  . 

When  this  is  substituted  in  Eq.  (A8-8),  the  value  of  p(U  =  0)  is  obtained,  and  is  found  to  be 
l/V  2NSnir . 

An  interesting  way  of  examining  p(U)  can  be  developed  as  follows.  Let 
0  (t)  *  0a  (t)  0b  (t) 


Thus, 


(t)  =' 


[  1  +  W"SN  t] 


(A8-9) 


0h  <0 - - — W7T 

b  [1  -  i'/SN~ t]  '  2 


(A8-9a) 


Since  it  is  known  that  p(U)  is  the  transform  of  0(t)  ,  it  must  be  the  convolution  of 
two  distribution  functions  p&  (V)  and  (W)  ,  namely, 
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in  which 


P(U)  =r“pa(V)  pb(U  -  V)  dV  , 

J  “00 

Pa  (V)  =27/  exp[— itV]  <pa  (t)  dt  , 

~Q0 

Pb(W)  =  2V r  exp[-itw]  0b(t)  dt 

*-)-  00 


(A8-10) 


(A8-  10a) 


(A8-10b) 


But  integrals  of  the  type  (A8-10a)  and  (-10b)  were  studieo  din.  Appendix  II.  In  partic¬ 
ular,  it  was  found  that  (W)  is  of  the  form  of  pQ  (W)  in  which  (2S)  has  been  replaced  by  ^SN  , 
and  similarly,  Pa  (V)  =  pQ  (— V)  with  the  same  substitution.  Thus,  V  exists  only  as  a  positive 
number  and  V  is  a  negative  number.  The  variable  desired,  U  ,  isaasum  of  a  positive  and  a 
negative  number  taken  from  the  same  distribution  of  magnitudes  (hoown  as  the  "chi-squared" 
distribution  in  statistics).  Thus,  the  form  of  Eq.  (A8-10)  may  be  chsanged  to  read 


P  (U)  =  f  PQ  (W)  pQ  (U  +  W)  dW 


(A8- 1 1) 


This  is  recognized  as  the  identical  form  of  the  autocorrelation  fundi  ion  of  U  for  the  probability 
density  distribution  p^  ! 

When  n  is  even  (n  —  1/  2  is  half  an  odd  integer)  the  Bess  sel  Function  reduces  to  a 
polynomial.  This  fact  can  be  obtained  as  a  result  of  the  autocor relaxation  process  indicated  in 
(A8-11)  -  namely,  if  pQ(W)  =  aWp  exp[-pW] ,  a  and  (3  being  writU-enfor  more  complicated 
constants, 

p(U)  =  f  azwp" 1  {U  +  w)p_1  exp[-2pW  -  PU]  dW  ,  U£0  ,  (A8-12) 

^O 

=  a2  expt-pU^f^wP'1  S  „Ck  wp'k_1  Uk  exp[-2pW]QdW  .  (A8-12a) 

Jo  k=o  p 

The  order  of  integration  and  summation  may  be  interchanged,  andtHhe  integration  resulting  is 
easy  to  perform.  Then, 


p(U)=  T  exp  4p£  /  1  f'K '  (2P)kljk-  u>oo  • 

(2prP'1  k=o  ^P  *  “ 

When  o  and  p  are  replaced  by  their  values  in  terms  of  S  and  W, 


(A8-  13) 


|  U  j  n  . 
exp  -  J — L.  -y  - 1 

nTSN  % 


(n  -  2  -  K) !  ( 2  ju| 


n(f):  ln*/SN  k=o  -k):k:  V^sn 


(A8-14) 


valid  for  all  U  when  n  is  even.  The  absolute -value  symbols  areirmcluded  because  (A8-12)  is 
derived  ortly  for  positive  U  .  The  knowledge  of  the  property  of  the 3  autocorrelation  function,  i.e.. 
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P(U)  =  p(-U),  is  used  to  extend  the  result  to  negative  values  of  U  - 

The  first  check  on  (A8-14)  occurs  for  n  =  2,  in  which  case 


p(U)  = 


1 

— - exp 

2\T 3N 


lul 

nTsn 


This  may  be  readily  verified  by  integration  of  (A8-7)  for  this  value  of  n  . 

As  a  final  inspection  of  the  properties  o'  p{U) ,  it  is  interesting  to  compare  certain 

properties  of  the  density  distribution  function  with  those  of  the  function  of  Appendix  II,  pQ(X)  . 

In  particular,  if  n/  2S  is  replaced  by  a  in  pQ  and  \TnS  is  replaced  by  a  in  p(U),  it  develops 

that  the  maximum  value  of  each  distribution  function  is  1/cW  2trn ,  while  the  variance  associated 
2 

wieh  each  is  na  .  They  differ  in  their  other  moment^  however,  in  as  much  as  (X)  is  defined 
only  for  magnitudes,  and  is  centered  about  an  average  na while  the  average  value  of  U  is 
zero. 
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